nixbot

1these 13 derivations will be built:2  /nix/store/rz0bid6v84j7zcrziygfb9g0rqg9lzl8-python3.13-nixbot.drv3  /nix/store/52kz094jfsf7qllqk9vnbm9zqrvd6xmk-unit-nixbot.service.drv4  /nix/store/bskzi850y5jdhp770rj6hv54r5crn6cb-system-units.drv5  /nix/store/491nyazfhjs66gxxca3ziv7l3dscw6y8-etc.drv6  /nix/store/0nk3rq22si6jgy8klxh1a32lnpcw24qn-activate.drv7  /nix/store/5fi8301mby2m920v1v9fc4davcrlvc7k-nixos-system-gitlab-test.drv8  /nix/store/n5dr5n3fjp4292p8f5h8gafwgimmfyf6-closure-info.drv9  /nix/store/h7d4gb58d5f6hijc4racrzl1kfx1ijf0-closure-info.drv10  /nix/store/z7llfy4gwsk2rxsvazx01gyi7vgsjxrs-run-nixos-vm.drv11  /nix/store/sgx896crlync48pvp0vjh89l8qni559s-nixos-vm.drv12  /nix/store/n507hkhd2gby9xww1wgz488611xhf253-driverConfiguration.json.drv13  /nix/store/2apqs5l3va9cq5wm8l074p46whxjszf2-nixos-test-driver-nixbot-gitlab.drv14  /nix/store/3pypn6q8vd32hi2zxy76pbq023wl284v-vm-test-run-nixbot-gitlab.drv15building '/nix/store/rz0bid6v84j7zcrziygfb9g0rqg9lzl8-python3.13-nixbot.drv' on 'ssh-ng://nix@eliza'16building '/nix/store/rz0bid6v84j7zcrziygfb9g0rqg9lzl8-python3.13-nixbot.drv'17python3.13-nixbot> Sourcing python-remove-tests-dir-hook18python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh19python3.13-nixbot> Sourcing python-remove-bin-bytecode-hook.sh20python3.13-nixbot> Sourcing pypa-build-hook21python3.13-nixbot> Using pypaBuildPhase22python3.13-nixbot> Sourcing python-runtime-deps-check-hook23python3.13-nixbot> Using pythonRuntimeDepsCheckHook24python3.13-nixbot> Sourcing pypa-install-hook25python3.13-nixbot> Using pypaInstallPhase26python3.13-nixbot> Sourcing python-imports-check-hook.sh27python3.13-nixbot> Using pythonImportsCheckPhase28python3.13-nixbot> Sourcing python-namespaces-hook29python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh30python3.13-nixbot> Running phase: unpackPhase31python3.13-nixbot> unpacking source archive /nix/store/4iihlf4yyshv2wm2nxjibgkpcy54jdp3-nixbot32python3.13-nixbot> source root is nixbot33python3.13-nixbot> setting SOURCE_DATE_EPOCH to timestamp 315619200 of file "nixbot/pyproject.toml"34python3.13-nixbot> Running phase: patchPhase35python3.13-nixbot> Running phase: updateAutotoolsGnuConfigScriptsPhase36python3.13-nixbot> Running phase: configurePhase37python3.13-nixbot> no configure script, doing nothing38python3.13-nixbot> Running phase: buildPhase39python3.13-nixbot> Executing pypaBuildPhase40python3.13-nixbot> Creating a wheel...41python3.13-nixbot> pypa build flags: --no-isolation --outdir dist/ --wheel42python3.13-nixbot> * Getting build dependencies for wheel...43python3.13-nixbot> * Building wheel...44python3.13-nixbot> Successfully built nixbot-0.0.1-py3-none-any.whl45python3.13-nixbot> Finished creating a wheel...46python3.13-nixbot> Finished executing pypaBuildPhase47python3.13-nixbot> Running phase: pythonRuntimeDepsCheckHook48python3.13-nixbot> Executing pythonRuntimeDepsCheck49python3.13-nixbot> Checking runtime dependencies for nixbot-0.0.1-py3-none-any.whl50python3.13-nixbot> Finished executing pythonRuntimeDepsCheck51python3.13-nixbot> Running phase: installPhase52python3.13-nixbot> Executing pypaInstallPhase53python3.13-nixbot> Successfully installed nixbot-0.0.1-py3-none-any.whl54python3.13-nixbot> Finished executing pypaInstallPhase55python3.13-nixbot> Running phase: pythonOutputDistPhase56python3.13-nixbot> Executing pythonOutputDistPhase57python3.13-nixbot> Finished executing pythonOutputDistPhase58python3.13-nixbot> Running phase: fixupPhase59python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/nyrdvq12bdkvj4xfasx1gp16s4mwnmy6-python3.13-nixbot60python3.13-nixbot> checking for references to /build/ in /nix/store/nyrdvq12bdkvj4xfasx1gp16s4mwnmy6-python3.13-nixbot...61python3.13-nixbot> patching script interpreter paths in /nix/store/nyrdvq12bdkvj4xfasx1gp16s4mwnmy6-python3.13-nixbot62python3.13-nixbot> stripping (with command strip and flags -S -p) in  /nix/store/nyrdvq12bdkvj4xfasx1gp16s4mwnmy6-python3.13-nixbot/lib /nix/store/nyrdvq12bdkvj4xfasx1gp16s4mwnmy6-python3.13-nixbot/bin63python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/6rwhw3mx7dkwqymkcbrvzs5cca6b6csr-python3.13-nixbot-dist64python3.13-nixbot> checking for references to /build/ in /nix/store/6rwhw3mx7dkwqymkcbrvzs5cca6b6csr-python3.13-nixbot-dist...65python3.13-nixbot> patching script interpreter paths in /nix/store/6rwhw3mx7dkwqymkcbrvzs5cca6b6csr-python3.13-nixbot-dist66python3.13-nixbot> Rewriting #!/nix/store/lqn6mbgzzdrqq2qkwddcmxj9z6amdd86-python3-3.13.13/bin/python3.13 to #!/nix/store/lqn6mbgzzdrqq2qkwddcmxj9z6amdd86-python3-3.13.1367python3.13-nixbot> wrapping `/nix/store/nyrdvq12bdkvj4xfasx1gp16s4mwnmy6-python3.13-nixbot/bin/nixbot'...68python3.13-nixbot> Executing pythonRemoveTestsDir69python3.13-nixbot> Finished executing pythonRemoveTestsDir70python3.13-nixbot> Running phase: pythonCatchConflictsPhase71python3.13-nixbot> Running phase: pythonRemoveBinBytecodePhase72python3.13-nixbot> Running phase: pythonImportsCheckPhase73python3.13-nixbot> Executing pythonImportsCheckPhase74building '/nix/store/52kz094jfsf7qllqk9vnbm9zqrvd6xmk-unit-nixbot.service.drv' on 'ssh-ng://nix@eliza'75building '/nix/store/52kz094jfsf7qllqk9vnbm9zqrvd6xmk-unit-nixbot.service.drv'76building '/nix/store/bskzi850y5jdhp770rj6hv54r5crn6cb-system-units.drv' on 'ssh-ng://nix@eliza'77building '/nix/store/bskzi850y5jdhp770rj6hv54r5crn6cb-system-units.drv'78building '/nix/store/491nyazfhjs66gxxca3ziv7l3dscw6y8-etc.drv' on 'ssh-ng://nix@eliza'79building '/nix/store/491nyazfhjs66gxxca3ziv7l3dscw6y8-etc.drv'80building '/nix/store/0nk3rq22si6jgy8klxh1a32lnpcw24qn-activate.drv' on 'ssh-ng://nix@eliza'81building '/nix/store/0nk3rq22si6jgy8klxh1a32lnpcw24qn-activate.drv'82building '/nix/store/5fi8301mby2m920v1v9fc4davcrlvc7k-nixos-system-gitlab-test.drv' on 'ssh-ng://nix@eliza'83building '/nix/store/5fi8301mby2m920v1v9fc4davcrlvc7k-nixos-system-gitlab-test.drv'84nixos-system-gitlab-test> structuredAttrs is enabled85building '/nix/store/n5dr5n3fjp4292p8f5h8gafwgimmfyf6-closure-info.drv' on 'ssh-ng://nix@eliza'86building '/nix/store/n5dr5n3fjp4292p8f5h8gafwgimmfyf6-closure-info.drv'87closure-info> structuredAttrs is enabled88building '/nix/store/h7d4gb58d5f6hijc4racrzl1kfx1ijf0-closure-info.drv' on 'ssh-ng://nix@eliza'89building '/nix/store/h7d4gb58d5f6hijc4racrzl1kfx1ijf0-closure-info.drv'90closure-info> structuredAttrs is enabled91building '/nix/store/z7llfy4gwsk2rxsvazx01gyi7vgsjxrs-run-nixos-vm.drv' on 'ssh-ng://nix@eliza'92building '/nix/store/z7llfy4gwsk2rxsvazx01gyi7vgsjxrs-run-nixos-vm.drv'93building '/nix/store/sgx896crlync48pvp0vjh89l8qni559s-nixos-vm.drv' on 'ssh-ng://nix@eliza'94building '/nix/store/sgx896crlync48pvp0vjh89l8qni559s-nixos-vm.drv'95building '/nix/store/n507hkhd2gby9xww1wgz488611xhf253-driverConfiguration.json.drv' on 'ssh-ng://nix@eliza'96building '/nix/store/n507hkhd2gby9xww1wgz488611xhf253-driverConfiguration.json.drv'97driverConfiguration.json> structuredAttrs is enabled98building '/nix/store/2apqs5l3va9cq5wm8l074p46whxjszf2-nixos-test-driver-nixbot-gitlab.drv' on 'ssh-ng://nix@eliza'99building '/nix/store/2apqs5l3va9cq5wm8l074p46whxjszf2-nixos-test-driver-nixbot-gitlab.drv'100nixos-test-driver-nixbot-gitlab> Running type check (enable/disable: config.skipTypeCheck)101nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck102nixos-test-driver-nixbot-gitlab> All checks passed!103nixos-test-driver-nixbot-gitlab> Linting test script (enable/disable: config.skipLint)104nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint105nixos-test-driver-nixbot-gitlab> All checks passed!106building '/nix/store/3pypn6q8vd32hi2zxy76pbq023wl284v-vm-test-run-nixbot-gitlab.drv' on 'ssh-ng://nix@eliza'107building '/nix/store/3pypn6q8vd32hi2zxy76pbq023wl284v-vm-test-run-nixbot-gitlab.drv'108vm-test-run-nixbot-gitlab> Machine state will be reset. To keep it, pass --keep-machine-state109vm-test-run-nixbot-gitlab> start all VLans110vm-test-run-nixbot-gitlab> (finished: start all VLans, in 0.00 seconds)111vm-test-run-nixbot-gitlab> Test will time out and terminate in 3600 seconds112vm-test-run-nixbot-gitlab> run the VM test script113vm-test-run-nixbot-gitlab> additionally exposed symbols:114vm-test-run-nixbot-gitlab>     gitlab,115vm-test-run-nixbot-gitlab>     vlan1,116vm-test-run-nixbot-gitlab>     start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh117vm-test-run-nixbot-gitlab> start all VMs118vm-test-run-nixbot-gitlab> gitlab: starting vm119vm-test-run-nixbot-gitlab> gitlab: QEMU running (pid 13)120vm-test-run-nixbot-gitlab> gitlab # Disk image does not exist, creating the virtualisation disk image...121vm-test-run-nixbot-gitlab> gitlab # Formatting '/build/vm-state-gitlab/tmp.gWl6QFdSuO', fmt=raw size=1073741824122vm-test-run-nixbot-gitlab> gitlab # mke2fs 1.47.3 (8-Jul-2025)123vm-test-run-nixbot-gitlab> gitlab # Discarding device blocks:      0/262144             done124vm-test-run-nixbot-gitlab> gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes125vm-test-run-nixbot-gitlab> gitlab # Filesystem UUID: 2b35cff4-c91d-46e0-972f-d5e44533ec81126vm-test-run-nixbot-gitlab> gitlab # Superblock backups stored on blocks:127vm-test-run-nixbot-gitlab> (finished: start all VMs, in 65.37 seconds)128vm-test-run-nixbot-gitlab> gitlab # 	32768, 98304, 163840, 229376129vm-test-run-nixbot-gitlab> gitlab: waiting for unit setup-gitlab.service130vm-test-run-nixbot-gitlab> gitlab # 131vm-test-run-nixbot-gitlab> gitlab: waiting for the VM to finish booting132vm-test-run-nixbot-gitlab> gitlab # Allocating group tables: 0/8   done133vm-test-run-nixbot-gitlab> gitlab # Writing inode tables: 0/8   done134vm-test-run-nixbot-gitlab> gitlab # Creating journal (8192 blocks): done135vm-test-run-nixbot-gitlab> gitlab # Writing superblocks and filesystem accounting information: 0/8   done136vm-test-run-nixbot-gitlab> gitlab # 137vm-test-run-nixbot-gitlab> gitlab # Virtualisation disk image created.138vm-test-run-nixbot-gitlab> gitlab # Creating Nix store image...139vm-test-run-nixbot-gitlab> gitlab # Created Nix store image.140vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]141vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Jun  1 15:51:08 UTC 2026142vm-test-run-nixbot-gitlab> gitlab # [    0.000000] KASLR enabled143vm-test-run-nixbot-gitlab> gitlab # [    0.000000] random: crng init done144vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Machine model: linux,dummy-virt145vm-test-run-nixbot-gitlab> gitlab # [    0.000000] efi: UEFI not found.146vm-test-run-nixbot-gitlab> gitlab # [    0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT147vm-test-run-nixbot-gitlab> gitlab # [    0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff]148vm-test-run-nixbot-gitlab> gitlab # [    0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf]149vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Zone ranges:150vm-test-run-nixbot-gitlab> gitlab # [    0.000000]   DMA      [mem 0x0000000040000000-0x00000000ffffffff]151vm-test-run-nixbot-gitlab> gitlab # [    0.000000]   DMA32    empty152vm-test-run-nixbot-gitlab> gitlab # [    0.000000]   Normal   [mem 0x0000000100000000-0x00000001bfffffff]153vm-test-run-nixbot-gitlab> gitlab # [    0.000000]   Device   empty154vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Movable zone start for each node155vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Early memory node ranges156vm-test-run-nixbot-gitlab> gitlab # [    0.000000]   node   0: [mem 0x0000000040000000-0x00000001bfffffff]157vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff]158vm-test-run-nixbot-gitlab> gitlab # [    0.000000] cma: Reserved 32 MiB at 0x00000000fe000000159vm-test-run-nixbot-gitlab> gitlab # [    0.000000] psci: probing for conduit method from DT.160vm-test-run-nixbot-gitlab> gitlab # [    0.000000] psci: PSCIv1.3 detected in firmware.161vm-test-run-nixbot-gitlab> gitlab # [    0.000000] psci: Using standard PSCI v0.2 function IDs162vm-test-run-nixbot-gitlab> gitlab # [    0.000000] psci: Trusted OS migration not required163vm-test-run-nixbot-gitlab> gitlab # [    0.000000] psci: SMC Calling Convention v1.1164vm-test-run-nixbot-gitlab> gitlab # [    0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)165vm-test-run-nixbot-gitlab> gitlab # [    0.000000] percpu: Embedded 76 pages/cpu s185944 r8192 d117160 u311296166vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Detected PIPT I-cache on CPU0167vm-test-run-nixbot-gitlab> gitlab # [    0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)168vm-test-run-nixbot-gitlab> gitlab # [    0.000000] CPU features: detected: GICv3 CPU interface169vm-test-run-nixbot-gitlab> gitlab # [    0.000000] CPU features: detected: Spectre-v4170vm-test-run-nixbot-gitlab> gitlab # [    0.000000] CPU features: detected: Spectre-BHB171vm-test-run-nixbot-gitlab> gitlab # [    0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38172vm-test-run-nixbot-gitlab> gitlab # [    0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23173vm-test-run-nixbot-gitlab> gitlab # [    0.000000] alternatives: applying boot alternatives174vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0p2r5ddxzfy3g3yjgwksl6hkvmn6gwff-nixos-system-gitlab-test/init regInfo=/nix/store/rh6q660nmdfyqnghhhmvjyajraqh14wi-closure-info/registration console=ttyAMA0,115200n8 console=tty0175vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Unknown kernel command line parameters "regInfo=/nix/store/rh6q660nmdfyqnghhhmvjyajraqh14wi-closure-info/registration", will be passed to user space.176vm-test-run-nixbot-gitlab> gitlab # [    0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes177vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)178vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)179vm-test-run-nixbot-gitlab> gitlab # [    0.000000] software IO TLB: area num 4.180vm-test-run-nixbot-gitlab> gitlab # [    0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB)181vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Fallback order for Node 0: 0182vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 1572864183vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Policy zone: Normal184vm-test-run-nixbot-gitlab> gitlab # [    0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off185vm-test-run-nixbot-gitlab> gitlab # [    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1186vm-test-run-nixbot-gitlab> gitlab # [    0.000000] allocated 12582912 bytes of page_ext187vm-test-run-nixbot-gitlab> gitlab # [    0.000000] ftrace: allocating 74648 entries in 292 pages188vm-test-run-nixbot-gitlab> gitlab # [    0.000000] ftrace: allocated 292 pages with 3 groups189vm-test-run-nixbot-gitlab> gitlab # [    0.000000] rcu: Hierarchical RCU implementation.190vm-test-run-nixbot-gitlab> gitlab # [    0.000000] rcu: 	RCU event tracing is enabled.191vm-test-run-nixbot-gitlab> gitlab # [    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.192vm-test-run-nixbot-gitlab> gitlab # [    0.000000] 	Trampoline variant of Tasks RCU enabled.193vm-test-run-nixbot-gitlab> gitlab # [    0.000000] 	Rude variant of Tasks RCU enabled.194vm-test-run-nixbot-gitlab> gitlab # [    0.000000] 	Tracing variant of Tasks RCU enabled.195vm-test-run-nixbot-gitlab> gitlab # [    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.196vm-test-run-nixbot-gitlab> gitlab # [    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4197vm-test-run-nixbot-gitlab> gitlab # [    0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.198vm-test-run-nixbot-gitlab> gitlab # [    0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.199vm-test-run-nixbot-gitlab> gitlab # [    0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.200vm-test-run-nixbot-gitlab> gitlab # [    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0201vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: 256 SPIs implemented202vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: 0 Extended SPIs implemented203vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Root IRQ handler: gic_handle_irq204vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI205vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0206vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000207vm-test-run-nixbot-gitlab> gitlab # [    0.000000] ITS [mem 0x08080000-0x0809ffff]208vm-test-run-nixbot-gitlab> gitlab # [    0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1)209vm-test-run-nixbot-gitlab> gitlab # [    0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1)210vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: using LPI property table @0x0000000100f30000211vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000212vm-test-run-nixbot-gitlab> gitlab # [    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.213vm-test-run-nixbot-gitlab> gitlab # [    0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).214vm-test-run-nixbot-gitlab> gitlab # [    0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns215vm-test-run-nixbot-gitlab> gitlab # [    0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns216vm-test-run-nixbot-gitlab> gitlab # [    0.000029] arm-pv: using stolen time PV217vm-test-run-nixbot-gitlab> gitlab # [    0.000396] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)218vm-test-run-nixbot-gitlab> gitlab # [    0.000572] Console: colour dummy device 80x25219vm-test-run-nixbot-gitlab> gitlab # [    0.000579] printk: legacy console [tty0] enabled220vm-test-run-nixbot-gitlab> gitlab # [    0.000763] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)221vm-test-run-nixbot-gitlab> gitlab # [    0.000770] pid_max: default: 32768 minimum: 301222vm-test-run-nixbot-gitlab> gitlab # [    0.000849] LSM: initializing lsm=capability,landlock,yama,bpf,ima223vm-test-run-nixbot-gitlab> gitlab # [    0.000982] landlock: Up and running.224vm-test-run-nixbot-gitlab> gitlab # [    0.000985] Yama: becoming mindful.225vm-test-run-nixbot-gitlab> gitlab # [    0.001393] LSM support for eBPF active226vm-test-run-nixbot-gitlab> gitlab # [    0.001610] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)227vm-test-run-nixbot-gitlab> gitlab # [    0.001720] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)228vm-test-run-nixbot-gitlab> gitlab # [    0.002895] cacheinfo: Unable to detect cache hierarchy for CPU 0229vm-test-run-nixbot-gitlab> gitlab # [    0.003559] rcu: Hierarchical SRCU implementation.230vm-test-run-nixbot-gitlab> gitlab # [    0.003563] rcu: 	Max phase no-delay instances is 1000.231vm-test-run-nixbot-gitlab> gitlab # [    0.003713] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level232vm-test-run-nixbot-gitlab> gitlab # [    0.004799] fsl-mc MSI: its@8080000 domain created233vm-test-run-nixbot-gitlab> gitlab # [    0.004892] EFI services will not be available.234vm-test-run-nixbot-gitlab> gitlab # [    0.005054] smp: Bringing up secondary CPUs ...235vm-test-run-nixbot-gitlab> gitlab # [    0.005703] Detected PIPT I-cache on CPU1236vm-test-run-nixbot-gitlab> gitlab # [    0.005804] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000237vm-test-run-nixbot-gitlab> gitlab # [    0.005935] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000238vm-test-run-nixbot-gitlab> gitlab # [    0.006065] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40]239vm-test-run-nixbot-gitlab> gitlab # [    0.007126] Detected PIPT I-cache on CPU2240vm-test-run-nixbot-gitlab> gitlab # [    0.007418] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000241vm-test-run-nixbot-gitlab> gitlab # [    0.007552] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000242vm-test-run-nixbot-gitlab> gitlab # [    0.007683] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40]243vm-test-run-nixbot-gitlab> gitlab # [    0.008680] Detected PIPT I-cache on CPU3244vm-test-run-nixbot-gitlab> gitlab # [    0.008800] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000245vm-test-run-nixbot-gitlab> gitlab # [    0.008933] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000246vm-test-run-nixbot-gitlab> gitlab # [    0.009067] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40]247vm-test-run-nixbot-gitlab> gitlab # [    0.009609] smp: Brought up 1 node, 4 CPUs248vm-test-run-nixbot-gitlab> gitlab # [    0.009636] SMP: Total of 4 processors activated.249vm-test-run-nixbot-gitlab> gitlab # [    0.009639] CPU: All CPU(s) started at EL1250vm-test-run-nixbot-gitlab> gitlab # [    0.009647] CPU features: detected: Branch Target Identification251vm-test-run-nixbot-gitlab> gitlab # [    0.009651] CPU features: detected: ARMv8.4 Translation Table Level252vm-test-run-nixbot-gitlab> gitlab # [    0.009653] CPU features: detected: Instruction cache invalidation not required for I/D coherence253vm-test-run-nixbot-gitlab> gitlab # [    0.009657] CPU features: detected: Data cache clean to the PoU not required for I/D coherence254vm-test-run-nixbot-gitlab> gitlab # [    0.009660] CPU features: detected: Common not Private translations255vm-test-run-nixbot-gitlab> gitlab # [    0.009663] CPU features: detected: CRC32 instructions256vm-test-run-nixbot-gitlab> gitlab # [    0.009665] CPU features: detected: Data cache clean to Point of Deep Persistence257vm-test-run-nixbot-gitlab> gitlab # [    0.009668] CPU features: detected: Data cache clean to Point of Persistence258vm-test-run-nixbot-gitlab> gitlab # [    0.009671] CPU features: detected: Data independent timing control (DIT)259vm-test-run-nixbot-gitlab> gitlab # [    0.009674] CPU features: detected: E0PD260vm-test-run-nixbot-gitlab> gitlab # [    0.009676] CPU features: detected: Enhanced Counter Virtualization261vm-test-run-nixbot-gitlab> gitlab # [    0.009679] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)262vm-test-run-nixbot-gitlab> gitlab # [    0.009682] CPU features: detected: Enhanced Virtualization Traps263vm-test-run-nixbot-gitlab> gitlab # [    0.009684] CPU features: detected: Fine Grained Traps264vm-test-run-nixbot-gitlab> gitlab # [    0.009688] CPU features: detected: Generic authentication (architected QARMA5 algorithm)265vm-test-run-nixbot-gitlab> gitlab # [    0.009691] CPU features: detected: RCpc load-acquire (LDAPR)266vm-test-run-nixbot-gitlab> gitlab # [    0.009694] CPU features: detected: LSE atomic instructions267vm-test-run-nixbot-gitlab> gitlab # [    0.009697] CPU features: detected: Privileged Access Never268vm-test-run-nixbot-gitlab> gitlab # [    0.009699] CPU features: detected: PMUv3269vm-test-run-nixbot-gitlab> gitlab # [    0.009701] CPU features: detected: RAS Extension Support270vm-test-run-nixbot-gitlab> gitlab # [    0.009704] CPU features: detected: RASv1p1 Extension Support271vm-test-run-nixbot-gitlab> gitlab # [    0.009706] CPU features: detected: Random Number Generator272vm-test-run-nixbot-gitlab> gitlab # [    0.009709] CPU features: detected: Speculation barrier (SB)273vm-test-run-nixbot-gitlab> gitlab # [    0.009711] CPU features: detected: Stage-2 Force Write-Back274vm-test-run-nixbot-gitlab> gitlab # [    0.009714] CPU features: detected: TLB range maintenance instructions275vm-test-run-nixbot-gitlab> gitlab # [    0.009717] CPU features: detected: Speculative Store Bypassing Safe (SSBS)276vm-test-run-nixbot-gitlab> gitlab # [    0.009843] alternatives: applying system-wide alternatives277vm-test-run-nixbot-gitlab> gitlab # [    0.012687] CPU features: detected: BBM Level 2 without TLB conflict abort278vm-test-run-nixbot-gitlab> gitlab # [    0.012890] Memory: 5965816K/6291456K available (24320K kernel code, 7086K rwdata, 26308K rodata, 4736K init, 1102K bss, 270904K reserved, 32768K cma-reserved)279vm-test-run-nixbot-gitlab> gitlab # [    0.014345] devtmpfs: initialized280vm-test-run-nixbot-gitlab> gitlab # [    0.018287] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns281vm-test-run-nixbot-gitlab> gitlab # [    0.018333] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)282vm-test-run-nixbot-gitlab> gitlab # [    0.018401] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).283vm-test-run-nixbot-gitlab> gitlab # [    0.018596] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL284vm-test-run-nixbot-gitlab> gitlab # [    0.018601] 0 pages in range for non-PLT usage285vm-test-run-nixbot-gitlab> gitlab # [    0.018602] 508336 pages in range for PLT usage286vm-test-run-nixbot-gitlab> gitlab # [    0.018742] pinctrl core: initialized pinctrl subsystem287vm-test-run-nixbot-gitlab> gitlab # [    0.019542] DMI not present or invalid.288vm-test-run-nixbot-gitlab> gitlab # [    0.022753] NET: Registered PF_NETLINK/PF_ROUTE protocol family289vm-test-run-nixbot-gitlab> gitlab # [    0.025447] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations290vm-test-run-nixbot-gitlab> gitlab # [    0.025716] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations291vm-test-run-nixbot-gitlab> gitlab # [    0.027563] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations292vm-test-run-nixbot-gitlab> gitlab # [    0.027584] audit: initializing netlink subsys (disabled)293vm-test-run-nixbot-gitlab> gitlab # [    0.027895] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1294vm-test-run-nixbot-gitlab> gitlab # [    0.028404] thermal_sys: Registered thermal governor 'fair_share'295vm-test-run-nixbot-gitlab> gitlab # [    0.028406] thermal_sys: Registered thermal governor 'bang_bang'296vm-test-run-nixbot-gitlab> gitlab # [    0.028411] thermal_sys: Registered thermal governor 'step_wise'297vm-test-run-nixbot-gitlab> gitlab # [    0.028413] thermal_sys: Registered thermal governor 'user_space'298vm-test-run-nixbot-gitlab> gitlab # [    0.028416] thermal_sys: Registered thermal governor 'power_allocator'299vm-test-run-nixbot-gitlab> gitlab # [    0.028494] cpuidle: using governor ladder300vm-test-run-nixbot-gitlab> gitlab # [    0.028557] cpuidle: using governor menu301vm-test-run-nixbot-gitlab> gitlab # [    0.028772] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.302vm-test-run-nixbot-gitlab> gitlab # [    0.028996] ASID allocator initialised with 65536 entries303vm-test-run-nixbot-gitlab> gitlab # [    0.030130] Serial: AMBA PL011 UART driver304vm-test-run-nixbot-gitlab> gitlab # [    0.035237] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1305vm-test-run-nixbot-gitlab> gitlab # [    0.035513] printk: console [ttyAMA0] enabled306vm-test-run-nixbot-gitlab> gitlab # [    0.047270] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages307vm-test-run-nixbot-gitlab> gitlab # [    0.047288] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page308vm-test-run-nixbot-gitlab> gitlab # [    0.047297] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages309vm-test-run-nixbot-gitlab> gitlab # [    0.047305] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page310vm-test-run-nixbot-gitlab> gitlab # [    0.047314] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages311vm-test-run-nixbot-gitlab> gitlab # [    0.047322] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page312vm-test-run-nixbot-gitlab> gitlab # [    0.047330] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages313vm-test-run-nixbot-gitlab> gitlab # [    0.047337] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page314vm-test-run-nixbot-gitlab> gitlab # [    0.059934] fbcon: Taking over console315vm-test-run-nixbot-gitlab> gitlab # [    0.059945] ACPI: Interpreter disabled.316vm-test-run-nixbot-gitlab> gitlab # [    0.060958] iommu: Default domain type: Translated317vm-test-run-nixbot-gitlab> gitlab # [    0.060975] iommu: DMA domain TLB invalidation policy: strict mode318vm-test-run-nixbot-gitlab> gitlab # [    0.075566] SCSI subsystem initialized319vm-test-run-nixbot-gitlab> gitlab # [    0.077745] usbcore: registered new interface driver usbfs320vm-test-run-nixbot-gitlab> gitlab # [    0.077884] usbcore: registered new interface driver hub321vm-test-run-nixbot-gitlab> gitlab # [    0.077964] usbcore: registered new device driver usb322vm-test-run-nixbot-gitlab> gitlab # [    0.079032] pps_core: LinuxPPS API ver. 1 registered323vm-test-run-nixbot-gitlab> gitlab # [    0.079044] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>324vm-test-run-nixbot-gitlab> gitlab # [    0.079066] PTP clock support registered325vm-test-run-nixbot-gitlab> gitlab # [    0.079167] EDAC MC: Ver: 3.0.0326vm-test-run-nixbot-gitlab> gitlab # [    0.080250] scmi_core: SCMI protocol bus registered327vm-test-run-nixbot-gitlab> gitlab # [    0.081611] FPGA manager framework328vm-test-run-nixbot-gitlab> gitlab # [    0.084376] vgaarb: loaded329vm-test-run-nixbot-gitlab> gitlab # [    0.088316] clocksource: Switched to clocksource arch_sys_counter330vm-test-run-nixbot-gitlab> gitlab # [    0.095591] VFS: Disk quotas dquot_6.6.0331vm-test-run-nixbot-gitlab> gitlab # [    0.095658] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)332vm-test-run-nixbot-gitlab> gitlab # [    0.096666] netfs: FS-Cache loaded333vm-test-run-nixbot-gitlab> gitlab # [    0.096943] pnp: PnP ACPI: disabled334vm-test-run-nixbot-gitlab> gitlab # [    0.108920] NET: Registered PF_INET protocol family335vm-test-run-nixbot-gitlab> gitlab # [    0.111592] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)336vm-test-run-nixbot-gitlab> gitlab # [    0.169953] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)337vm-test-run-nixbot-gitlab> gitlab # [    0.170019] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)338vm-test-run-nixbot-gitlab> gitlab # [    0.170075] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)339vm-test-run-nixbot-gitlab> gitlab # [    0.170456] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)340vm-test-run-nixbot-gitlab> gitlab # [    0.171327] TCP: Hash tables configured (established 65536 bind 65536)341vm-test-run-nixbot-gitlab> gitlab # [    0.171468] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)342vm-test-run-nixbot-gitlab> gitlab # [    0.171869] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)343vm-test-run-nixbot-gitlab> gitlab # [    0.172387] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)344vm-test-run-nixbot-gitlab> gitlab # [    0.172636] NET: Registered PF_UNIX/PF_LOCAL protocol family345vm-test-run-nixbot-gitlab> gitlab # [    0.172665] NET: Registered PF_XDP protocol family346vm-test-run-nixbot-gitlab> gitlab # [    0.172683] PCI: CLS 0 bytes, default 64347vm-test-run-nixbot-gitlab> gitlab # [    0.173098] Trying to unpack rootfs image as initramfs...348vm-test-run-nixbot-gitlab> gitlab # [    0.188432] kvm [1]: HYP mode not available349vm-test-run-nixbot-gitlab> gitlab # [    0.279176] Initialise system trusted keyrings350vm-test-run-nixbot-gitlab> gitlab # [    0.279531] workingset: timestamp_bits=42 max_order=21 bucket_order=0351vm-test-run-nixbot-gitlab> gitlab # [    0.280822] squashfs: version 4.0 (2009/01/31) Phillip Lougher352vm-test-run-nixbot-gitlab> gitlab # [    0.280994] 9p: Installing v9fs 9p2000 file system support353vm-test-run-nixbot-gitlab> gitlab # [    0.317826] Key type asymmetric registered354vm-test-run-nixbot-gitlab> gitlab # [    0.317855] Asymmetric key parser 'x509' registered355vm-test-run-nixbot-gitlab> gitlab # [    0.317968] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)356vm-test-run-nixbot-gitlab> gitlab # [    0.318265] io scheduler mq-deadline registered357vm-test-run-nixbot-gitlab> gitlab # [    0.318279] io scheduler kyber registered358vm-test-run-nixbot-gitlab> gitlab # [    0.335455] pl061_gpio 9030000.pl061: PL061 GPIO chip registered359vm-test-run-nixbot-gitlab> gitlab # [    0.340378] ledtrig-cpu: registered to indicate activity on CPUs360vm-test-run-nixbot-gitlab> gitlab # [    0.341950] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:361vm-test-run-nixbot-gitlab> gitlab # [    0.342009] pci-host-generic 4010000000.pcie:       IO 0x003eff0000..0x003effffff -> 0x0000000000362vm-test-run-nixbot-gitlab> gitlab # [    0.342078] pci-host-generic 4010000000.pcie:      MEM 0x0010000000..0x003efeffff -> 0x0010000000363vm-test-run-nixbot-gitlab> gitlab # [    0.342099] pci-host-generic 4010000000.pcie:      MEM 0x8000000000..0xffffffffff -> 0x8000000000364vm-test-run-nixbot-gitlab> gitlab # [    0.342162] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits365vm-test-run-nixbot-gitlab> gitlab # [    0.342214] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]366vm-test-run-nixbot-gitlab> gitlab # [    0.342486] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00367vm-test-run-nixbot-gitlab> gitlab # [    0.342505] pci_bus 0000:00: root bus resource [bus 00-ff]368vm-test-run-nixbot-gitlab> gitlab # [    0.342537] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]369vm-test-run-nixbot-gitlab> gitlab # [    0.342548] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]370vm-test-run-nixbot-gitlab> gitlab # [    0.342557] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]371vm-test-run-nixbot-gitlab> gitlab # [    0.342826] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint372vm-test-run-nixbot-gitlab> gitlab # [    0.344308] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint373vm-test-run-nixbot-gitlab> gitlab # [    0.345011] pci 0000:00:01.0: BAR 0 [io  0x0000-0x001f]374vm-test-run-nixbot-gitlab> gitlab # [    0.345048] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]375vm-test-run-nixbot-gitlab> gitlab # [    0.345131] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]376vm-test-run-nixbot-gitlab> gitlab # [    0.345167] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]377vm-test-run-nixbot-gitlab> gitlab # [    0.346722] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint378vm-test-run-nixbot-gitlab> gitlab # [    0.347349] pci 0000:00:02.0: BAR 0 [io  0x0000-0x001f]379vm-test-run-nixbot-gitlab> gitlab # [    0.347384] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]380vm-test-run-nixbot-gitlab> gitlab # [    0.347467] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]381vm-test-run-nixbot-gitlab> gitlab # [    0.349126] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint382vm-test-run-nixbot-gitlab> gitlab # [    0.349766] pci 0000:00:03.0: BAR 0 [io  0x0000-0x001f]383vm-test-run-nixbot-gitlab> gitlab # [    0.349803] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]384vm-test-run-nixbot-gitlab> gitlab # [    0.349885] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]385vm-test-run-nixbot-gitlab> gitlab # [    0.351462] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint386vm-test-run-nixbot-gitlab> gitlab # [    0.352091] pci 0000:00:04.0: BAR 0 [io  0x0000-0x001f]387vm-test-run-nixbot-gitlab> gitlab # [    0.352127] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]388vm-test-run-nixbot-gitlab> gitlab # [    0.352208] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]389vm-test-run-nixbot-gitlab> gitlab # [    0.353869] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint390vm-test-run-nixbot-gitlab> gitlab # [    0.354523] pci 0000:00:05.0: BAR 0 [io  0x0000-0x007f]391vm-test-run-nixbot-gitlab> gitlab # [    0.354560] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]392vm-test-run-nixbot-gitlab> gitlab # [    0.354642] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]393vm-test-run-nixbot-gitlab> gitlab # [    0.356262] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint394vm-test-run-nixbot-gitlab> gitlab # [    0.356950] pci 0000:00:06.0: BAR 0 [io  0x0000-0x007f]395vm-test-run-nixbot-gitlab> gitlab # [    0.356986] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]396vm-test-run-nixbot-gitlab> gitlab # [    0.357068] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]397vm-test-run-nixbot-gitlab> gitlab # [    0.358706] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint398vm-test-run-nixbot-gitlab> gitlab # [    0.359353] pci 0000:00:07.0: BAR 0 [io  0x0000-0x001f]399vm-test-run-nixbot-gitlab> gitlab # [    0.359388] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]400vm-test-run-nixbot-gitlab> gitlab # [    0.359469] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]401vm-test-run-nixbot-gitlab> gitlab # [    0.359506] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]402vm-test-run-nixbot-gitlab> gitlab # [    0.361221] pci 0000:00:08.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint403vm-test-run-nixbot-gitlab> gitlab # [    0.361885] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]404vm-test-run-nixbot-gitlab> gitlab # [    0.361966] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]405vm-test-run-nixbot-gitlab> gitlab # [    0.363566] pci 0000:00:09.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint406vm-test-run-nixbot-gitlab> gitlab # [    0.364198] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]407vm-test-run-nixbot-gitlab> gitlab # [    0.364280] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]408vm-test-run-nixbot-gitlab> gitlab # [    0.365713] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint409vm-test-run-nixbot-gitlab> gitlab # [    0.366320] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]410vm-test-run-nixbot-gitlab> gitlab # [    0.367189] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint411vm-test-run-nixbot-gitlab> gitlab # [    0.367912] pci 0000:00:0b.0: BAR 0 [io  0x0000-0x003f]412vm-test-run-nixbot-gitlab> gitlab # [    0.367949] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]413vm-test-run-nixbot-gitlab> gitlab # [    0.368030] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]414vm-test-run-nixbot-gitlab> gitlab # [    0.369694] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint415vm-test-run-nixbot-gitlab> gitlab # [    0.370329] pci 0000:00:0c.0: BAR 0 [io  0x0000-0x001f]416vm-test-run-nixbot-gitlab> gitlab # [    0.370364] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]417vm-test-run-nixbot-gitlab> gitlab # [    0.370445] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]418vm-test-run-nixbot-gitlab> gitlab # [    0.372369] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned419vm-test-run-nixbot-gitlab> gitlab # [    0.372387] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned420vm-test-run-nixbot-gitlab> gitlab # [    0.372398] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned421vm-test-run-nixbot-gitlab> gitlab # [    0.372542] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned422vm-test-run-nixbot-gitlab> gitlab # [    0.372676] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned423vm-test-run-nixbot-gitlab> gitlab # [    0.372818] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned424vm-test-run-nixbot-gitlab> gitlab # [    0.372953] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned425vm-test-run-nixbot-gitlab> gitlab # [    0.373115] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned426vm-test-run-nixbot-gitlab> gitlab # [    0.373247] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned427vm-test-run-nixbot-gitlab> gitlab # [    0.373392] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned428vm-test-run-nixbot-gitlab> gitlab # [    0.373554] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned429vm-test-run-nixbot-gitlab> gitlab # [    0.373687] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned430vm-test-run-nixbot-gitlab> gitlab # [    0.373861] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned431vm-test-run-nixbot-gitlab> gitlab # [    0.374002] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned432vm-test-run-nixbot-gitlab> gitlab # [    0.374057] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned433vm-test-run-nixbot-gitlab> gitlab # [    0.374119] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned434vm-test-run-nixbot-gitlab> gitlab # [    0.374173] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned435vm-test-run-nixbot-gitlab> gitlab # [    0.374236] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned436vm-test-run-nixbot-gitlab> gitlab # [    0.374291] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned437vm-test-run-nixbot-gitlab> gitlab # [    0.374345] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned438vm-test-run-nixbot-gitlab> gitlab # [    0.374400] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned439vm-test-run-nixbot-gitlab> gitlab # [    0.374468] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned440vm-test-run-nixbot-gitlab> gitlab # [    0.374525] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned441vm-test-run-nixbot-gitlab> gitlab # [    0.374581] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned442vm-test-run-nixbot-gitlab> gitlab # [    0.374638] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned443vm-test-run-nixbot-gitlab> gitlab # [    0.374692] pci 0000:00:05.0: BAR 0 [io  0x1000-0x107f]: assigned444vm-test-run-nixbot-gitlab> gitlab # [    0.374745] pci 0000:00:06.0: BAR 0 [io  0x1080-0x10ff]: assigned445vm-test-run-nixbot-gitlab> gitlab # [    0.374811] pci 0000:00:0b.0: BAR 0 [io  0x1100-0x113f]: assigned446vm-test-run-nixbot-gitlab> gitlab # [    0.374865] pci 0000:00:01.0: BAR 0 [io  0x1140-0x115f]: assigned447vm-test-run-nixbot-gitlab> gitlab # [    0.374927] pci 0000:00:02.0: BAR 0 [io  0x1160-0x117f]: assigned448vm-test-run-nixbot-gitlab> gitlab # [    0.374982] pci 0000:00:03.0: BAR 0 [io  0x1180-0x119f]: assigned449vm-test-run-nixbot-gitlab> gitlab # [    0.375036] pci 0000:00:04.0: BAR 0 [io  0x11a0-0x11bf]: assigned450vm-test-run-nixbot-gitlab> gitlab # [    0.375091] pci 0000:00:07.0: BAR 0 [io  0x11c0-0x11df]: assigned451vm-test-run-nixbot-gitlab> gitlab # [    0.375145] pci 0000:00:0c.0: BAR 0 [io  0x11e0-0x11ff]: assigned452vm-test-run-nixbot-gitlab> gitlab # [    0.375209] pci_bus 0000:00: resource 4 [io  0x0000-0xffff]453vm-test-run-nixbot-gitlab> gitlab # [    0.375224] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]454vm-test-run-nixbot-gitlab> gitlab # [    0.375233] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]455vm-test-run-nixbot-gitlab> gitlab # [    0.379490] pci 0000:00:0a.0: enabling device (0000 -> 0002)456vm-test-run-nixbot-gitlab> gitlab # [    0.414422] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)457vm-test-run-nixbot-gitlab> gitlab # [    0.420243] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)458vm-test-run-nixbot-gitlab> gitlab # [    0.426768] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)459vm-test-run-nixbot-gitlab> gitlab # [    0.433192] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)460vm-test-run-nixbot-gitlab> gitlab # [    0.439134] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)461vm-test-run-nixbot-gitlab> gitlab # [    0.445706] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)462vm-test-run-nixbot-gitlab> gitlab # [    0.451680] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)463vm-test-run-nixbot-gitlab> gitlab # [    0.458256] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)464vm-test-run-nixbot-gitlab> gitlab # [    0.464161] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)465vm-test-run-nixbot-gitlab> gitlab # [    0.470477] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)466vm-test-run-nixbot-gitlab> gitlab # [    0.477150] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)467vm-test-run-nixbot-gitlab> gitlab # [    0.491837] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled468vm-test-run-nixbot-gitlab> gitlab # [    0.496346] msm_serial: driver initialized469vm-test-run-nixbot-gitlab> gitlab # [    0.496706] SuperH (H)SCI(F) driver initialized470vm-test-run-nixbot-gitlab> gitlab # [    0.496830] STM32 USART driver initialized471vm-test-run-nixbot-gitlab> gitlab # [    0.535720] loop: module loaded472vm-test-run-nixbot-gitlab> gitlab # [    0.536073] virtio_blk virtio4: 4/0/0 default/read/poll queues473vm-test-run-nixbot-gitlab> gitlab # [    0.540172] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)474vm-test-run-nixbot-gitlab> gitlab # [    0.543512] virtio_blk virtio5: 4/0/0 default/read/poll queues475vm-test-run-nixbot-gitlab> gitlab # [    0.547038] virtio_blk virtio5: [vdb] 11826944 512-byte logical blocks (6.06 GB/5.64 GiB)476vm-test-run-nixbot-gitlab> gitlab # [    0.550982] megasas: 07.734.00.00-rc1477vm-test-run-nixbot-gitlab> gitlab # [    0.552292] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]478vm-test-run-nixbot-gitlab> gitlab # [    0.555678] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000479vm-test-run-nixbot-gitlab> gitlab # [    0.555780] Intel/Sharp Extended Query Table at 0x0031480vm-test-run-nixbot-gitlab> gitlab # [    0.558810] Using buffer write method481vm-test-run-nixbot-gitlab> gitlab # [    0.558857] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]482vm-test-run-nixbot-gitlab> gitlab # [    0.562426] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000483vm-test-run-nixbot-gitlab> gitlab # [    0.562449] Intel/Sharp Extended Query Table at 0x0031484vm-test-run-nixbot-gitlab> gitlab # [    0.566597] Using buffer write method485vm-test-run-nixbot-gitlab> gitlab # [    0.566620] Concatenating MTD devices:486vm-test-run-nixbot-gitlab> gitlab # [    0.566624] (0): "0.flash"487vm-test-run-nixbot-gitlab> gitlab # [    0.566628] (1): "0.flash"488vm-test-run-nixbot-gitlab> gitlab # [    0.566632] into device "0.flash"489vm-test-run-nixbot-gitlab> gitlab # [    0.600332] Freeing initrd memory: 25372K490vm-test-run-nixbot-gitlab> gitlab # [    0.618703] tun: Universal TUN/TAP device driver, 1.6491vm-test-run-nixbot-gitlab> gitlab # [    0.628582] thunder_xcv, ver 1.0492vm-test-run-nixbot-gitlab> gitlab # [    0.628671] thunder_bgx, ver 1.0493vm-test-run-nixbot-gitlab> gitlab # [    0.628729] nicpf, ver 1.0494vm-test-run-nixbot-gitlab> gitlab # [    0.630490] e1000: Intel(R) PRO/1000 Network Driver495vm-test-run-nixbot-gitlab> gitlab # [    0.630503] e1000: Copyright (c) 1999-2006 Intel Corporation.496vm-test-run-nixbot-gitlab> gitlab # [    0.630568] e1000e: Intel(R) PRO/1000 Network Driver497vm-test-run-nixbot-gitlab> gitlab # [    0.630582] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.498vm-test-run-nixbot-gitlab> gitlab # [    0.630651] igb: Intel(R) Gigabit Ethernet Network Driver499vm-test-run-nixbot-gitlab> gitlab # [    0.630660] igb: Copyright (c) 2007-2014 Intel Corporation.500vm-test-run-nixbot-gitlab> gitlab # [    0.630748] igbvf: Intel(R) Gigabit Virtual Function Network Driver501vm-test-run-nixbot-gitlab> gitlab # [    0.630759] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.502vm-test-run-nixbot-gitlab> gitlab # [    0.631154] sky2: driver version 1.30503vm-test-run-nixbot-gitlab> gitlab # [    0.635593] usbcore: registered new interface driver usb-storage504vm-test-run-nixbot-gitlab> gitlab # [    0.635854] usbcore: registered new interface driver usbserial_generic505vm-test-run-nixbot-gitlab> gitlab # [    0.635987] usbserial: USB Serial support registered for generic506vm-test-run-nixbot-gitlab> gitlab # [    0.636015] ehci-pci 0000:00:0a.0: EHCI Host Controller507vm-test-run-nixbot-gitlab> gitlab # [    0.636084] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1508vm-test-run-nixbot-gitlab> gitlab # [    0.636844] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000509vm-test-run-nixbot-gitlab> gitlab # [    0.637687] hv_vmbus: registering driver hyperv_keyboard510vm-test-run-nixbot-gitlab> gitlab # [    0.639925] rtc-pl031 9010000.pl031: registered as rtc0511vm-test-run-nixbot-gitlab> gitlab # [    0.639976] rtc-pl031 9010000.pl031: setting system clock to 2026-06-29T11:56:30 UTC (1782734190)512vm-test-run-nixbot-gitlab> gitlab # [    0.640770] i2c_dev: i2c /dev entries driver513vm-test-run-nixbot-gitlab> gitlab # [    0.647792] sdhci: Secure Digital Host Controller Interface driver514vm-test-run-nixbot-gitlab> gitlab # [    0.647802] sdhci: Copyright(c) Pierre Ossman515vm-test-run-nixbot-gitlab> gitlab # [    0.648446] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00516vm-test-run-nixbot-gitlab> gitlab # [    0.648475] Synopsys Designware Multimedia Card Interface Driver517vm-test-run-nixbot-gitlab> gitlab # [    0.649233] hub 1-0:1.0: USB hub found518vm-test-run-nixbot-gitlab> gitlab # [    0.649268] hub 1-0:1.0: 6 ports detected519vm-test-run-nixbot-gitlab> gitlab # [    0.649447] sdhci-pltfm: SDHCI platform and OF driver helper520vm-test-run-nixbot-gitlab> gitlab # [    0.653197] hid: raw HID events driver (C) Jiri Kosina521vm-test-run-nixbot-gitlab> gitlab # [    0.653952] usbcore: registered new interface driver usbhid522vm-test-run-nixbot-gitlab> gitlab # [    0.653961] usbhid: USB HID core driver523vm-test-run-nixbot-gitlab> gitlab # [    0.693841] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available524vm-test-run-nixbot-gitlab> gitlab # [    0.698870] drop_monitor: Initializing network drop monitor service525vm-test-run-nixbot-gitlab> gitlab # [    0.699367] NET: Registered PF_INET6 protocol family526vm-test-run-nixbot-gitlab> gitlab # [    0.701779] Segment Routing with IPv6527vm-test-run-nixbot-gitlab> gitlab # [    0.701812] In-situ OAM (IOAM) with IPv6528vm-test-run-nixbot-gitlab> gitlab # [    0.702162] NET: Registered PF_PACKET protocol family529vm-test-run-nixbot-gitlab> gitlab # [    0.702459] 9pnet: Installing 9P2000 support530vm-test-run-nixbot-gitlab> gitlab # [    0.707222] Key type dns_resolver registered531vm-test-run-nixbot-gitlab> gitlab # [    0.726105] registered taskstats version 1532vm-test-run-nixbot-gitlab> gitlab # [    0.726720] Loading compiled-in X.509 certificates533vm-test-run-nixbot-gitlab> gitlab # [    0.753945] Demotion targets for Node 0: null534vm-test-run-nixbot-gitlab> gitlab # [    0.754177] Key type .fscrypt registered535vm-test-run-nixbot-gitlab> gitlab # [    0.754185] Key type fscrypt-provisioning registered536vm-test-run-nixbot-gitlab> gitlab # [    0.754404] ima: No TPM chip found, activating TPM-bypass!537vm-test-run-nixbot-gitlab> gitlab # [    0.754430] ima: Allocated hash algorithm: sha1538vm-test-run-nixbot-gitlab> gitlab # [    0.754462] ima: No architecture policies found539vm-test-run-nixbot-gitlab> gitlab # [    0.755821] input: gpio-keys as /devices/platform/gpio-keys/input/input0540vm-test-run-nixbot-gitlab> gitlab # [    0.789486] clk: Disabling unused clocks541vm-test-run-nixbot-gitlab> gitlab # [    0.789526] PM: genpd: Disabling unused power domains542vm-test-run-nixbot-gitlab> gitlab # [    0.798135] Freeing unused kernel memory: 4736K543vm-test-run-nixbot-gitlab> gitlab # [    0.798465] Run /init as init process544vm-test-run-nixbot-gitlab> gitlab # [    0.843226] systemd[1]: Successfully made /usr/ read-only.545vm-test-run-nixbot-gitlab> gitlab # [    0.900449] usb 1-1: new high-speed USB device number 2 using ehci-pci546vm-test-run-nixbot-gitlab> gitlab # [    1.058699] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1547vm-test-run-nixbot-gitlab> gitlab # [    1.145071] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0548vm-test-run-nixbot-gitlab> gitlab # [    1.178298] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)549vm-test-run-nixbot-gitlab> gitlab # [    1.178360] systemd[1]: Detected virtualization qemu.550vm-test-run-nixbot-gitlab> gitlab # [    1.178429] systemd[1]: Detected architecture arm64.551vm-test-run-nixbot-gitlab> gitlab # [    1.178444] systemd[1]: Running in initrd.552vm-test-run-nixbot-gitlab> gitlab # [    1.179428] systemd[1]: Initializing machine ID from random generator.553vm-test-run-nixbot-gitlab> gitlab # [    1.179772] systemd[1]: Hostname set to <gitlab>.554vm-test-run-nixbot-gitlab> gitlab # [    1.272376] usb 1-2: new high-speed USB device number 3 using ehci-pci555vm-test-run-nixbot-gitlab> gitlab # [    1.331774] systemd[1]: Queued start job for default target Initrd Default Target.556vm-test-run-nixbot-gitlab> gitlab # [    1.426554] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2557vm-test-run-nixbot-gitlab> gitlab # [    1.427152] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0558vm-test-run-nixbot-gitlab> gitlab # [    1.572972] systemd[1]: Created slice Slice /system/modprobe.559vm-test-run-nixbot-gitlab> gitlab # [    1.573633] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.560vm-test-run-nixbot-gitlab> gitlab # [    1.573711] systemd[1]: Expecting device /dev/disk/by-label/nix-store...561vm-test-run-nixbot-gitlab> gitlab # [    1.573742] systemd[1]: Expecting device /dev/disk/by-label/nixos...562vm-test-run-nixbot-gitlab> gitlab # [    1.573790] systemd[1]: Reached target Path Units.563vm-test-run-nixbot-gitlab> gitlab # [    1.573825] systemd[1]: Reached target Slice Units.564vm-test-run-nixbot-gitlab> gitlab # [    1.573863] systemd[1]: Reached target Swaps.565vm-test-run-nixbot-gitlab> gitlab # [    1.573899] systemd[1]: Reached target Timer Units.566vm-test-run-nixbot-gitlab> gitlab # [    1.574238] systemd[1]: Listening on D-Bus System Message Bus Socket.567vm-test-run-nixbot-gitlab> gitlab # [    1.574563] systemd[1]: Listening on Journal Socket (/dev/log).568vm-test-run-nixbot-gitlab> gitlab # [    1.574948] systemd[1]: Listening on Journal Sockets.569vm-test-run-nixbot-gitlab> gitlab # [    1.575251] systemd[1]: Listening on udev Control Socket.570vm-test-run-nixbot-gitlab> gitlab # [    1.575440] systemd[1]: Listening on udev Kernel Socket.571vm-test-run-nixbot-gitlab> gitlab # [    1.575482] systemd[1]: Reached target Socket Units.572vm-test-run-nixbot-gitlab> gitlab # [    1.583178] systemd[1]: Starting Create List of Static Device Nodes...573vm-test-run-nixbot-gitlab> gitlab # [    1.590580] systemd[1]: Starting Load Kernel Module 9pnet_virtio...574vm-test-run-nixbot-gitlab> gitlab # [    1.590721] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs575vm-test-run-nixbot-gitlab> gitlab # [    1.598437] systemd[1]: Mounting Kernel Configuration File System...576vm-test-run-nixbot-gitlab> gitlab # [    1.620901] systemd[1]: Starting Journal Service...577vm-test-run-nixbot-gitlab> gitlab # [    1.628379] systemd[1]: Starting Load Kernel Modules...578vm-test-run-nixbot-gitlab> gitlab # [    1.628510] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki579vm-test-run-nixbot-gitlab> gitlab # [    1.635341] systemd[1]: Starting Coldplug All udev Devices...580vm-test-run-nixbot-gitlab> gitlab # [    1.651329] systemd[1]: Finished Create List of Static Device Nodes.581vm-test-run-nixbot-gitlab> gitlab # [    1.654285] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.582vm-test-run-nixbot-gitlab> gitlab # [    1.661786] systemd[1]: Finished Load Kernel Module 9pnet_virtio.583vm-test-run-nixbot-gitlab> gitlab # [    1.671310] [drm] pci: virtio-gpu-pci detected at 0000:00:09.0584vm-test-run-nixbot-gitlab> gitlab # [    1.671722] [drm] features: -virgl +edid -resource_blob -host_visible585vm-test-run-nixbot-gitlab> gitlab # [    1.671727] [drm] features: -context_init586vm-test-run-nixbot-gitlab> gitlab # [    1.676996] systemd-journald[154]: Collecting audit messages is disabled.587vm-test-run-nixbot-gitlab> gitlab # [    1.681425] [drm] number of scanouts: 1588vm-test-run-nixbot-gitlab> gitlab # [    1.681432] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.589vm-test-run-nixbot-gitlab> gitlab # [    1.681470] [drm] number of cap sets: 0590vm-test-run-nixbot-gitlab> gitlab # [    1.681690] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev591vm-test-run-nixbot-gitlab> gitlab # [    1.684728] systemd[1]: Mounted Kernel Configuration File System.592vm-test-run-nixbot-gitlab> gitlab # [    1.686469] virtio-pci 0000:00:09.0: [drm] Registered 1 planes with drm panic593vm-test-run-nixbot-gitlab> gitlab # [    1.686480] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:09.0 on minor 0594vm-test-run-nixbot-gitlab> gitlab # [    1.698139] Console: switching to colour frame buffer device 160x50595vm-test-run-nixbot-gitlab> gitlab # [    1.706485] virtio-pci 0000:00:09.0: [drm] fb0: virtio_gpudrmfb frame buffer device596vm-test-run-nixbot-gitlab> gitlab # [    1.715166] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...597vm-test-run-nixbot-gitlab> gitlab # [    1.744770] systemd[1]: Finished Load Kernel Modules.598vm-test-run-nixbot-gitlab> gitlab # [    1.752603] systemd[1]: Starting Apply Kernel Variables...599vm-test-run-nixbot-gitlab> gitlab # [    1.764245] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.600vm-test-run-nixbot-gitlab> gitlab # [    1.763027] systemd-modules-load[156]: Using 4 probe threads601vm-test-run-nixbot-gitlab> gitlab # [    1.764794] systemd-modules-load[156]: Module 'virtio_balloon' is built in602vm-test-run-nixbot-gitlab> gitlab # [    1.765991] systemd-modules-load[156]: Module 'virtio_console' is built in603vm-test-run-nixbot-gitlab> gitlab # [    1.767121] systemd-modules-load[156]: Module 'virtio_rng' is built in604vm-test-run-nixbot-gitlab> gitlab # [    1.768243] systemd-modules-load[156]: Inserted module 'dm_mod'605vm-test-run-nixbot-gitlab> gitlab # [    1.769382] systemd-modules-load[156]: Inserted module 'virtio_gpu'606vm-test-run-nixbot-gitlab> gitlab # [    1.789523] systemd[1]: Starting Create Static Device Nodes in /dev...607vm-test-run-nixbot-gitlab> gitlab # [    1.790281] systemd[1]: Started Journal Service.608vm-test-run-nixbot-gitlab> gitlab # [    1.813195] systemd[1]: Finished Apply Kernel Variables.609vm-test-run-nixbot-gitlab> gitlab # [    1.818905] systemd[1]: Finished Create Static Device Nodes in /dev.610vm-test-run-nixbot-gitlab> gitlab # [    1.821916] systemd[1]: Reached target Preparation for Local File Systems.611vm-test-run-nixbot-gitlab> gitlab # [    1.825046] systemd[1]: Reached target Local File Systems.612vm-test-run-nixbot-gitlab> gitlab # [    1.827988] systemd[1]: Starting Create System Files and Directories...613vm-test-run-nixbot-gitlab> gitlab # [    1.845367] systemd[1]: Starting Rule-based Manager for Device Events and Files...614vm-test-run-nixbot-gitlab> gitlab # [    1.863140] systemd[1]: Finished Create System Files and Directories.615vm-test-run-nixbot-gitlab> gitlab # [    1.901434] systemd-udevd[191]: Using default interface naming scheme 'v260'.616vm-test-run-nixbot-gitlab> gitlab # [    1.927238] systemd[1]: Started Rule-based Manager for Device Events and Files.617vm-test-run-nixbot-gitlab> gitlab # [    1.964143] systemd[1]: Starting Virtual Console Setup...618vm-test-run-nixbot-gitlab> gitlab # [    2.025162] systemd-vconsole-setup[207]: Configuration of first virtual console was skipped, ignoring remaining ones.619vm-test-run-nixbot-gitlab> gitlab # [    2.032546] systemd[1]: Finished Virtual Console Setup.620vm-test-run-nixbot-gitlab> gitlab # [    2.364768] systemd[1]: Finished Coldplug All udev Devices.621vm-test-run-nixbot-gitlab> gitlab # [    2.367373] systemd[1]: Reached target System Initialization.622vm-test-run-nixbot-gitlab> gitlab # [    2.370028] systemd[1]: Reached target Basic System.623vm-test-run-nixbot-gitlab> gitlab # [    2.563299] (udev-worker)[213]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.624vm-test-run-nixbot-gitlab> gitlab # [    2.570572] (udev-worker)[213]: Network interface NamePolicy= disabled on kernel command line.625vm-test-run-nixbot-gitlab> gitlab # [    2.574146] (udev-worker)[218]: Network interface NamePolicy= disabled on kernel command line.626vm-test-run-nixbot-gitlab> gitlab # [    2.585602] systemd[1]: Found device /dev/disk/by-label/nix-store.627vm-test-run-nixbot-gitlab> gitlab # [    2.594662] systemd[1]: Found device /dev/disk/by-label/nixos.628vm-test-run-nixbot-gitlab> gitlab # [    2.613274] systemd[1]: Reached target Initrd Root Device.629vm-test-run-nixbot-gitlab> gitlab # [    2.618140] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...630vm-test-run-nixbot-gitlab> gitlab # [    2.651851] systemd-fsck[232]: nixos: clean, 12/65536 files, 13019/262144 blocks631vm-test-run-nixbot-gitlab> gitlab # [    2.657743] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.632vm-test-run-nixbot-gitlab> gitlab # [    2.659593] systemd[1]: Mounting /sysroot...633vm-test-run-nixbot-gitlab> gitlab # [    2.692073] EXT4-fs (vda): mounted filesystem 2b35cff4-c91d-46e0-972f-d5e44533ec81 r/w with ordered data mode. Quota mode: none.634vm-test-run-nixbot-gitlab> gitlab # [    2.692926] systemd[1]: Mounted /sysroot.635vm-test-run-nixbot-gitlab> gitlab # [    2.693680] systemd[1]: Reached target Initrd Root File System.636vm-test-run-nixbot-gitlab> gitlab # [    2.696726] systemd[1]: Starting Mountpoints Configured in the Real Root...637vm-test-run-nixbot-gitlab> gitlab # [    2.719071] systemd-sysroot-fstab-check[243]: /sysroot should be mounted in the initrd, will request daemon-reload.638vm-test-run-nixbot-gitlab> gitlab # [    2.721331] systemd[1]: Reload requested from client PID 243 ('systemd-sysroot') (unit initrd-parse-etc.service)...639vm-test-run-nixbot-gitlab> gitlab # [    2.725805] systemd[1]: Reloading...640vm-test-run-nixbot-gitlab> gitlab # [    3.021653] systemd[1]: Reloading finished in 300 ms.641vm-test-run-nixbot-gitlab> gitlab # [    3.055124] systemd-sysroot-fstab-check[243]: Requesting initrd-fs.target/start/replace...642vm-test-run-nixbot-gitlab> gitlab # [    3.139201] systemd-sysroot-fstab-check[243]: Requesting swap.target/start/replace...643vm-test-run-nixbot-gitlab> gitlab # [    3.142138] systemd[1]: Starting Load Kernel Module 9pnet_virtio...644vm-test-run-nixbot-gitlab> gitlab # [    3.145160] systemd[1]: initrd-parse-etc.service: Deactivated successfully.645vm-test-run-nixbot-gitlab> gitlab # [    3.158291] systemd[1]: Finished Mountpoints Configured in the Real Root.646vm-test-run-nixbot-gitlab> gitlab # [    3.159365] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.647vm-test-run-nixbot-gitlab> gitlab # [    3.162297] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.648vm-test-run-nixbot-gitlab> gitlab # [    3.164168] systemd[1]: Finished Load Kernel Module 9pnet_virtio.649vm-test-run-nixbot-gitlab> gitlab # [    3.201629] (udev-worker)[196]: mtd0ro: Failed to find and pin callout binary "/nix/store/3x4plcsrj2r4mmzis46wrmyka8g4jhi8-systemd-260.1/lib/udev/mtd_probe": No such file or directory650vm-test-run-nixbot-gitlab> gitlab # [    3.204599] (udev-worker)[196]: mtd0ro: /nix/store/3x4plcsrj2r4mmzis46wrmyka8g4jhi8-systemd-260.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory651vm-test-run-nixbot-gitlab> gitlab # [    3.208988] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.652vm-test-run-nixbot-gitlab> gitlab # [    3.210106] systemd[1]: Stopped Virtual Console Setup.653vm-test-run-nixbot-gitlab> gitlab # [    3.211838] systemd[1]: Stopping Virtual Console Setup...654vm-test-run-nixbot-gitlab> gitlab # [    3.214509] systemd[1]: Starting Virtual Console Setup...655vm-test-run-nixbot-gitlab> gitlab # [    3.232914] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.656vm-test-run-nixbot-gitlab> gitlab # [    3.234626] systemd[1]: Stopped Virtual Console Setup.657vm-test-run-nixbot-gitlab> gitlab # [    3.240054] systemd[1]: Starting Virtual Console Setup...658vm-test-run-nixbot-gitlab> gitlab # [    3.259410] systemd-vconsole-setup[458]: Configuration of first virtual console was skipped, ignoring remaining ones.659vm-test-run-nixbot-gitlab> gitlab # [    3.262748] systemd[1]: Finished Virtual Console Setup.660vm-test-run-nixbot-gitlab> gitlab # [    3.651915] systemd[1]: Mounting /sysroot/nix/.ro-store...661vm-test-run-nixbot-gitlab> gitlab # [    3.659526] systemd[1]: Mounting /sysroot/nix/.rw-store...662vm-test-run-nixbot-gitlab> gitlab # [    3.677332] systemd[1]: Mounting /sysroot/run...663vm-test-run-nixbot-gitlab> gitlab # [    3.688230] systemd[1]: Mounting /sysroot/tmp/shared...664vm-test-run-nixbot-gitlab> gitlab # [    3.698831] systemd[1]: Mounting /sysroot/tmp/xchg...665vm-test-run-nixbot-gitlab> gitlab # [    3.714421] erofs (device vdb): mounted with root inode @ nid 128.666vm-test-run-nixbot-gitlab> gitlab # [    3.726597] systemd[1]: Mounted /sysroot/nix/.ro-store.667vm-test-run-nixbot-gitlab> gitlab # [    3.729467] systemd[1]: Mounted /sysroot/nix/.rw-store.668vm-test-run-nixbot-gitlab> gitlab # [    3.732088] systemd[1]: Mounted /sysroot/run.669vm-test-run-nixbot-gitlab> gitlab # [    3.734093] systemd[1]: Mounted /sysroot/tmp/shared.670vm-test-run-nixbot-gitlab> gitlab # [    3.736426] systemd[1]: Mounted /sysroot/tmp/xchg.671vm-test-run-nixbot-gitlab> gitlab # [    3.748492] systemd[1]: Starting rw-sysroot-nix-store.service...672vm-test-run-nixbot-gitlab> gitlab # [    3.780317] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.673vm-test-run-nixbot-gitlab> gitlab # [    3.785155] systemd[1]: Finished rw-sysroot-nix-store.service.674vm-test-run-nixbot-gitlab> gitlab # [    3.801681] systemd[1]: Mounting /sysroot/nix/store...675vm-test-run-nixbot-gitlab> gitlab # [    3.861956] systemd[1]: Mounted /sysroot/nix/store.676vm-test-run-nixbot-gitlab> gitlab # [    3.864345] systemd[1]: Reached target Initrd File Systems.677vm-test-run-nixbot-gitlab> gitlab # [    3.869815] systemd[1]: Starting Find NixOS closure...678vm-test-run-nixbot-gitlab> gitlab # [    3.877390] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...679vm-test-run-nixbot-gitlab> gitlab # [    3.922224] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.680vm-test-run-nixbot-gitlab> gitlab # [    3.940708] systemd[1]: Finished Find NixOS closure.681vm-test-run-nixbot-gitlab> gitlab # [    3.942966] systemd[1]: Reached target Initrd Default Target.682vm-test-run-nixbot-gitlab> gitlab # [    3.948335] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...683vm-test-run-nixbot-gitlab> gitlab # [    4.005296] systemd[1]: Stopped target Initrd Default Target.684vm-test-run-nixbot-gitlab> gitlab # [    4.008155] systemd[1]: Stopped target Basic System.685vm-test-run-nixbot-gitlab> gitlab # [    4.010410] systemd[1]: Stopped target Initrd Root Device.686vm-test-run-nixbot-gitlab> gitlab # [    4.012932] systemd[1]: Stopped target Path Units.687vm-test-run-nixbot-gitlab> gitlab # [    4.015142] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.688vm-test-run-nixbot-gitlab> gitlab # [    4.018569] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.689vm-test-run-nixbot-gitlab> gitlab # [    4.022013] systemd[1]: Stopped target Slice Units.690vm-test-run-nixbot-gitlab> gitlab # [    4.024271] systemd[1]: Stopped target Socket Units.691vm-test-run-nixbot-gitlab> gitlab # [    4.026489] systemd[1]: Stopped target System Initialization.692vm-test-run-nixbot-gitlab> gitlab # [    4.029137] systemd[1]: Stopped target Swaps.693vm-test-run-nixbot-gitlab> gitlab # [    4.030985] systemd[1]: Stopped target Timer Units.694vm-test-run-nixbot-gitlab> gitlab # [    4.033045] systemd[1]: dbus.socket: Deactivated successfully.695vm-test-run-nixbot-gitlab> gitlab # [    4.035202] systemd[1]: Closed D-Bus System Message Bus Socket.696vm-test-run-nixbot-gitlab> gitlab # [    4.037412] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.697vm-test-run-nixbot-gitlab> gitlab # [    4.039902] systemd[1]: Stopped Find NixOS closure.698vm-test-run-nixbot-gitlab> gitlab # [    4.041680] systemd[1]: Starting Load Kernel Module 9pnet_virtio...699vm-test-run-nixbot-gitlab> gitlab # [    4.043603] systemd[1]: Starting rw-sysroot-nix-store.service...700vm-test-run-nixbot-gitlab> gitlab # [    4.045540] systemd[1]: systemd-sysctl.service: Deactivated successfully.701vm-test-run-nixbot-gitlab> gitlab # [    4.047485] systemd[1]: Stopped Apply Kernel Variables.702vm-test-run-nixbot-gitlab> gitlab # [    4.051365] systemd[1]: systemd-modules-load.service: Deactivated successfully.703vm-test-run-nixbot-gitlab> gitlab # [    4.053478] systemd[1]: Stopped Load Kernel Modules.704vm-test-run-nixbot-gitlab> gitlab # [    4.061098] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.705vm-test-run-nixbot-gitlab> gitlab # [    4.063995] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.706vm-test-run-nixbot-gitlab> gitlab # [    4.069792] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.707vm-test-run-nixbot-gitlab> gitlab # [    4.072973] systemd[1]: Stopped Create System Files and Directories.708vm-test-run-nixbot-gitlab> gitlab # [    4.077847] systemd[1]: Stopped target Local File Systems.709vm-test-run-nixbot-gitlab> gitlab # [    4.080633] systemd[1]: Stopped target Preparation for Local File Systems.710vm-test-run-nixbot-gitlab> gitlab # [    4.083837] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.711vm-test-run-nixbot-gitlab> gitlab # [    4.087174] systemd[1]: Stopped Coldplug All udev Devices.712vm-test-run-nixbot-gitlab> gitlab # [    4.089894] systemd[1]: Stopping Rule-based Manager for Device Events and Files...713vm-test-run-nixbot-gitlab> gitlab # [    4.093212] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.714vm-test-run-nixbot-gitlab> gitlab # [    4.096443] systemd[1]: Stopped Virtual Console Setup.715vm-test-run-nixbot-gitlab> gitlab # [    4.103845] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.716vm-test-run-nixbot-gitlab> gitlab # [    4.107493] systemd[1]: Finished Load Kernel Module 9pnet_virtio.717vm-test-run-nixbot-gitlab> gitlab # [    4.114759] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.718vm-test-run-nixbot-gitlab> gitlab # [    4.118435] systemd[1]: Finished rw-sysroot-nix-store.service.719vm-test-run-nixbot-gitlab> gitlab # [    4.128004] systemd[1]: systemd-udevd.service: Deactivated successfully.720vm-test-run-nixbot-gitlab> gitlab # [    4.133182] systemd[1]: Stopped Rule-based Manager for Device Events and Files.721vm-test-run-nixbot-gitlab> gitlab # [    4.136366] systemd[1]: systemd-udevd.service: Consumed 2.370s CPU time over 2.282s wall clock time, 32.2M memory peak.722vm-test-run-nixbot-gitlab> gitlab # [    4.151876] systemd[1]: initrd-cleanup.service: Deactivated successfully.723vm-test-run-nixbot-gitlab> gitlab # [    4.157234] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.724vm-test-run-nixbot-gitlab> gitlab # [    4.170843] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.725vm-test-run-nixbot-gitlab> gitlab # [    4.174104] systemd[1]: Closed udev Control Socket.726vm-test-run-nixbot-gitlab> gitlab # [    4.186307] systemd[1]: Starting Cleanup udev Database...727vm-test-run-nixbot-gitlab> gitlab # [    4.188859] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.728vm-test-run-nixbot-gitlab> gitlab # [    4.192219] systemd[1]: Stopped Create Static Device Nodes in /dev.729vm-test-run-nixbot-gitlab> gitlab # [    4.194961] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.730vm-test-run-nixbot-gitlab> gitlab # [    4.198802] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.731vm-test-run-nixbot-gitlab> gitlab # [    4.203233] systemd[1]: kmod-static-nodes.service: Deactivated successfully.732vm-test-run-nixbot-gitlab> gitlab # [    4.206338] systemd[1]: Stopped Create List of Static Device Nodes.733vm-test-run-nixbot-gitlab> gitlab # [    4.286079] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.734vm-test-run-nixbot-gitlab> gitlab # [    4.289517] systemd[1]: Finished Cleanup udev Database.735vm-test-run-nixbot-gitlab> gitlab # [    4.298438] systemd[1]: Reached target Switch Root.736vm-test-run-nixbot-gitlab> gitlab # [    4.306797] systemd[1]: Starting NixOS Activation...737vm-test-run-nixbot-gitlab> gitlab # [    4.487178] initrd-nixos-activation-start[559]: booting system configuration /nix/store/0p2r5ddxzfy3g3yjgwksl6hkvmn6gwff-nixos-system-gitlab-test738vm-test-run-nixbot-gitlab> gitlab # [    4.556326] initrd-nixos-activation-start[559]: running activation script...739vm-test-run-nixbot-gitlab> gitlab # [    4.889429] initrd-nixos-activation-start[582]: setting up /etc...740vm-test-run-nixbot-gitlab> gitlab # [    5.031119] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.741vm-test-run-nixbot-gitlab> gitlab # [    5.035873] systemd[1]: Finished NixOS Activation.742vm-test-run-nixbot-gitlab> gitlab # [    5.052463] systemd[1]: Starting Switch Root...743vm-test-run-nixbot-gitlab> gitlab # [    5.104778] systemd[1]: Switching root.744vm-test-run-nixbot-gitlab> gitlab # [    5.737039] systemd-journald[154]: Received SIGTERM from PID 1 (systemd).745vm-test-run-nixbot-gitlab> gitlab # [    6.442265] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)746vm-test-run-nixbot-gitlab> gitlab # [    6.442408] systemd[1]: Detected virtualization qemu.747vm-test-run-nixbot-gitlab> gitlab # [    6.442498] systemd[1]: Detected architecture arm64.748vm-test-run-nixbot-gitlab> gitlab # [    6.442673] systemd[1]: Detected first boot.749vm-test-run-nixbot-gitlab> gitlab # [    6.444240] systemd[1]: Initializing machine ID from random generator.750vm-test-run-nixbot-gitlab> gitlab # [    6.692465] systemd[1]: bpf-restrict-fs: LSM BPF program attached751vm-test-run-nixbot-gitlab> gitlab # [    6.799386] systemd[1]: Applying preset policy.752vm-test-run-nixbot-gitlab> gitlab # [    7.045389] systemd[1]: Populated /etc with preset unit settings.753vm-test-run-nixbot-gitlab> gitlab # [    7.214101] systemd[1]: initrd-switch-root.service: Deactivated successfully.754vm-test-run-nixbot-gitlab> gitlab # [    7.214776] systemd[1]: Stopped initrd-switch-root.service.755vm-test-run-nixbot-gitlab> gitlab # [    7.218277] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.756vm-test-run-nixbot-gitlab> gitlab # [    7.224765] systemd[1]: Created slice Slice /system/getty.757vm-test-run-nixbot-gitlab> gitlab # [    7.229241] systemd[1]: Created slice GitLab DevOps Platform Slice.758vm-test-run-nixbot-gitlab> gitlab # [    7.233108] systemd[1]: Created slice User and Session Slice.759vm-test-run-nixbot-gitlab> gitlab # [    7.234602] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.760vm-test-run-nixbot-gitlab> gitlab # [    7.235925] systemd[1]: Started Forward Password Requests to Wall Directory Watch.761vm-test-run-nixbot-gitlab> gitlab # [    7.237483] systemd[1]: Expecting device /dev/hvc0...762vm-test-run-nixbot-gitlab> gitlab # [    7.238583] systemd[1]: Expecting device /dev/ttyAMA0...763vm-test-run-nixbot-gitlab> gitlab # [    7.239679] systemd[1]: Reached target Local Encrypted Volumes.764vm-test-run-nixbot-gitlab> gitlab # [    7.240814] systemd[1]: Stopped target initrd-fs.target.765vm-test-run-nixbot-gitlab> gitlab # [    7.241874] systemd[1]: Stopped target initrd-root-fs.target.766vm-test-run-nixbot-gitlab> gitlab # [    7.242836] systemd[1]: Stopped target initrd-switch-root.target.767vm-test-run-nixbot-gitlab> gitlab # [    7.243816] systemd[1]: Reached target Virtual Machines and Containers.768vm-test-run-nixbot-gitlab> gitlab # [    7.244888] systemd[1]: Reached target Path Units.769vm-test-run-nixbot-gitlab> gitlab # [    7.245865] systemd[1]: Reached target Remote File Systems.770vm-test-run-nixbot-gitlab> gitlab # [    7.246808] systemd[1]: Reached target Slice Units.771vm-test-run-nixbot-gitlab> gitlab # [    7.247754] systemd[1]: Reached target Swaps.772vm-test-run-nixbot-gitlab> gitlab # [    7.251747] systemd[1]: Listening on Process Core Dump Socket.773vm-test-run-nixbot-gitlab> gitlab # [    7.254733] systemd[1]: Listening on Credential Encryption/Decryption.774vm-test-run-nixbot-gitlab> gitlab # [    7.262904] systemd[1]: Starting Journal Log Access Socket...775vm-test-run-nixbot-gitlab> gitlab # [    7.265010] systemd[1]: Listening on Journal Audit Socket.776vm-test-run-nixbot-gitlab> gitlab # [    7.266595] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.777vm-test-run-nixbot-gitlab> gitlab # [    7.267362] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki778vm-test-run-nixbot-gitlab> gitlab # [    7.268620] systemd[1]: Listening on udev Control Socket.779vm-test-run-nixbot-gitlab> gitlab # [    7.308811] systemd[1]: Mounting Huge Pages File System...780vm-test-run-nixbot-gitlab> gitlab # [    7.314164] systemd[1]: Mounting POSIX Message Queue File System...781vm-test-run-nixbot-gitlab> gitlab # [    7.320241] systemd[1]: Mounting Kernel Debug File System...782vm-test-run-nixbot-gitlab> gitlab # [    7.329395] systemd[1]: Mounting Kernel Trace File System...783vm-test-run-nixbot-gitlab> gitlab # [    7.335285] systemd[1]: Starting Create List of Static Device Nodes...784vm-test-run-nixbot-gitlab> gitlab # [    7.352591] systemd[1]: Starting Load Kernel Module 9pnet_virtio...785vm-test-run-nixbot-gitlab> gitlab # [    7.357214] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs786vm-test-run-nixbot-gitlab> gitlab # [    7.368416] systemd[1]: Mounting Kernel Configuration File System...787vm-test-run-nixbot-gitlab> gitlab # [    7.369519] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm788vm-test-run-nixbot-gitlab> gitlab # [    7.370405] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore789vm-test-run-nixbot-gitlab> gitlab # [    7.381966] systemd[1]: Starting Load Kernel Module fuse...790vm-test-run-nixbot-gitlab> gitlab # [    7.383024] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67791vm-test-run-nixbot-gitlab> gitlab # [    7.394450] systemd[1]: Starting Journal Service...792vm-test-run-nixbot-gitlab> gitlab # [    7.399206] systemd[1]: Starting Load Kernel Modules...793vm-test-run-nixbot-gitlab> gitlab # [    7.408803] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...794vm-test-run-nixbot-gitlab> gitlab # [    7.412845] systemd[1]: Starting Remount Root and Kernel File Systems...795vm-test-run-nixbot-gitlab> gitlab # [    7.416599] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki796vm-test-run-nixbot-gitlab> gitlab # [    7.431760] systemd[1]: Starting Coldplug All udev Devices...797vm-test-run-nixbot-gitlab> gitlab # [    7.433472] fuse: init (API version 7.45)798vm-test-run-nixbot-gitlab> gitlab # [    7.454529] systemd[1]: Listening on Journal Log Access Socket.799vm-test-run-nixbot-gitlab> gitlab # [    7.455519] systemd[1]: Mounted Huge Pages File System.800vm-test-run-nixbot-gitlab> gitlab # [    7.456460] systemd[1]: Mounted POSIX Message Queue File System.801vm-test-run-nixbot-gitlab> gitlab # [    7.457343] systemd[1]: Mounted Kernel Debug File System.802vm-test-run-nixbot-gitlab> gitlab # [    7.458193] systemd[1]: Mounted Kernel Trace File System.803vm-test-run-nixbot-gitlab> gitlab # [    7.459450] systemd[1]: Finished Create List of Static Device Nodes.804vm-test-run-nixbot-gitlab> gitlab # [    7.460674] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.805vm-test-run-nixbot-gitlab> gitlab # [    7.461645] systemd[1]: Finished Load Kernel Module 9pnet_virtio.806vm-test-run-nixbot-gitlab> gitlab # [    7.462523] systemd[1]: Mounted Kernel Configuration File System.807vm-test-run-nixbot-gitlab> gitlab # [    7.463505] systemd[1]: modprobe@fuse.service: Deactivated successfully.808vm-test-run-nixbot-gitlab> gitlab # [    7.464573] systemd[1]: Finished Load Kernel Module fuse.809vm-test-run-nixbot-gitlab> gitlab # [    7.465824] systemd[1]: Finished Load Kernel Modules.810vm-test-run-nixbot-gitlab> gitlab # [    7.471393] EXT4-fs (vda): re-mounted 2b35cff4-c91d-46e0-972f-d5e44533ec81.811vm-test-run-nixbot-gitlab> gitlab # [    7.471855] systemd-journald[791]: Collecting audit messages is enabled.812vm-test-run-nixbot-gitlab> gitlab # [    7.476477] systemd[1]: Mounting FUSE Control File System...813vm-test-run-nixbot-gitlab> gitlab # [    7.486331] systemd[1]: Starting Firewall...814vm-test-run-nixbot-gitlab> gitlab # [    7.497721] systemd[1]: Queued start job for default target Multi-User System.815vm-test-run-nixbot-gitlab> gitlab # [    7.500069] systemd[1]: Unnecessary job was removed for /dev/vdb.816vm-test-run-nixbot-gitlab> gitlab # [    7.501981] systemd[1]: systemd-journald.service: Deactivated successfully.817vm-test-run-nixbot-gitlab> gitlab # [    7.504068] systemd-modules-load[792]: Using 3 probe threads818vm-test-run-nixbot-gitlab> gitlab # [    7.508861] systemd[1]: Starting Apply Kernel Variables...819vm-test-run-nixbot-gitlab> gitlab # [    7.507788] systemd-modules-load[792]: Module 'atkbd' is built in820vm-test-run-nixbot-gitlab> gitlab # [    7.509697] systemd-modules-load[792]: Module 'loop' is built in821vm-test-run-nixbot-gitlab> gitlab # [    7.511476] systemd-modules-load[792]: Inserted module 'tls'[    7.516837] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...822vm-test-run-nixbot-gitlab> gitlab # 823vm-test-run-nixbot-gitlab> gitlab # [    7.516617] systemd-oomd[793]: No swap; memory pressure usage will be degraded[    7.522134] systemd[1]: Started Journal Service.824vm-test-run-nixbot-gitlab> gitlab # 825vm-test-run-nixbot-gitlab> gitlab # [    7.522038] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.826vm-test-run-nixbot-gitlab> gitlab # [    7.524358] systemd[1]: Finished Remount Root and Kernel File Systems.827vm-test-run-nixbot-gitlab> gitlab # [    7.527116] systemd[1]: Mounted FUSE Control File System.828vm-test-run-nixbot-gitlab> gitlab # [    7.538470] systemd[1]: Finished Apply Kernel Variables.829vm-test-run-nixbot-gitlab> gitlab # [    7.544479] systemd[1]: Starting Flush Journal to Persistent Storage...830vm-test-run-nixbot-gitlab> gitlab # [    7.547046] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore831vm-test-run-nixbot-gitlab> gitlab # [    7.549328] systemd[1]: Starting Load/Save OS Random Seed...832vm-test-run-nixbot-gitlab> gitlab # [    7.550266] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki833vm-test-run-nixbot-gitlab> gitlab # [    7.571513] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.834vm-test-run-nixbot-gitlab> gitlab # [    7.576452] systemd[1]: Starting Create Static Device Nodes in /dev...835vm-test-run-nixbot-gitlab> gitlab # [    7.583620] systemd-journald[791]: Received client request to flush runtime journal.836vm-test-run-nixbot-gitlab> gitlab # [    7.620225] systemd[1]: Finished Create Static Device Nodes in /dev.837vm-test-run-nixbot-gitlab> gitlab # [    7.621834] systemd[1]: Reached target Preparation for Local File Systems.838vm-test-run-nixbot-gitlab> gitlab # [    7.623505] systemd[1]: Starting Rule-based Manager for Device Events and Files...839vm-test-run-nixbot-gitlab> gitlab # [    7.626006] systemd[1]: Finished Load/Save OS Random Seed.840vm-test-run-nixbot-gitlab> gitlab # [    7.627166] systemd[1]: Reached target First Boot Complete.841vm-test-run-nixbot-gitlab> gitlab # [    7.628425] systemd[1]: Finished Flush Journal to Persistent Storage.842vm-test-run-nixbot-gitlab> gitlab # [    7.674875] systemd-udevd[845]: Using default interface naming scheme 'v260'.843vm-test-run-nixbot-gitlab> gitlab # [    7.737438] systemd[1]: Started Rule-based Manager for Device Events and Files.844vm-test-run-nixbot-gitlab> gitlab # [    7.880146] systemd[1]: Finished Firewall.845vm-test-run-nixbot-gitlab> gitlab # [    8.178329] systemd[1]: Finished Coldplug All udev Devices.846vm-test-run-nixbot-gitlab> gitlab # [    8.213830] systemd[1]: Mounting /run/wrappers...847vm-test-run-nixbot-gitlab> gitlab # [    8.246247] systemd[1]: Mounted /run/wrappers.848vm-test-run-nixbot-gitlab> gitlab # [    8.248447] systemd[1]: Reached target Local File Systems.849vm-test-run-nixbot-gitlab> gitlab # [    8.250981] systemd[1]: Listening on Boot Loader Control Service Socket.850vm-test-run-nixbot-gitlab> gitlab # [    8.256459] systemd[1]: Starting register-nix-paths.service...851vm-test-run-nixbot-gitlab> gitlab # [    8.266250] systemd[1]: Starting Create SUID/SGID Wrappers...852vm-test-run-nixbot-gitlab> gitlab # [    8.269497] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.853vm-test-run-nixbot-gitlab> gitlab # [    8.288861] systemd[1]: Starting Save Transient machine-id to Disk...854vm-test-run-nixbot-gitlab> gitlab # [    8.300108] systemd[1]: Starting Create System Files and Directories...855vm-test-run-nixbot-gitlab> gitlab # [    8.370981] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.856vm-test-run-nixbot-gitlab> gitlab # [    8.373487] systemd[1]: Finished Save Transient machine-id to Disk.857vm-test-run-nixbot-gitlab> gitlab # [    8.376392] systemd[1]: Finished Create System Files and Directories.858vm-test-run-nixbot-gitlab> gitlab # [    8.385806] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs859vm-test-run-nixbot-gitlab> gitlab # [    8.392148] systemd[1]: Starting Rebuild Journal Catalog...860vm-test-run-nixbot-gitlab> gitlab # [    8.395546] systemd[1]: Starting Record System Boot/Shutdown in UTMP...861vm-test-run-nixbot-gitlab> gitlab # [    8.400339] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.862vm-test-run-nixbot-gitlab> gitlab # [    8.413266] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped.863vm-test-run-nixbot-gitlab> gitlab # [    8.422064] systemd[1]: Finished Record System Boot/Shutdown in UTMP.864vm-test-run-nixbot-gitlab> gitlab # [    8.444610] systemd[1]: Finished Rebuild Journal Catalog.865vm-test-run-nixbot-gitlab> gitlab # [    8.448129] systemd[1]: Starting Update is Completed...866vm-test-run-nixbot-gitlab> gitlab # [    8.462439] (udev-worker)[910]: Network interface NamePolicy= disabled on kernel command line.867vm-test-run-nixbot-gitlab> gitlab # [    8.477755] systemd[1]: Finished Update is Completed.868vm-test-run-nixbot-gitlab> gitlab # [    8.479952] (udev-worker)[892]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.869vm-test-run-nixbot-gitlab> gitlab # [    8.485750] (udev-worker)[892]: Network interface NamePolicy= disabled on kernel command line.870vm-test-run-nixbot-gitlab> gitlab # [    8.516617] systemd[1]: Condition check resulted in Virtio network device being skipped.871vm-test-run-nixbot-gitlab> gitlab # [    8.518831] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore872vm-test-run-nixbot-gitlab> gitlab # [    8.520864] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.873vm-test-run-nixbot-gitlab> gitlab # [    8.522204] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67874vm-test-run-nixbot-gitlab> gitlab # [    8.524631] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore875vm-test-run-nixbot-gitlab> gitlab # [    8.526278] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki876vm-test-run-nixbot-gitlab> gitlab # [    8.527619] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki877vm-test-run-nixbot-gitlab> gitlab # [    8.532985] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.878vm-test-run-nixbot-gitlab> gitlab # [    8.534950] systemd[1]: Finished Create SUID/SGID Wrappers.879vm-test-run-nixbot-gitlab> gitlab # [    8.535814] systemd[1]: suid-sgid-wrappers.service: Consumed 178ms CPU time over 268ms wall clock time, 6.4M memory peak, 2.2M read from disk.880vm-test-run-nixbot-gitlab> gitlab # [    8.565363] mousedev: PS/2 mouse device common for all mice881vm-test-run-nixbot-gitlab> gitlab # [    8.807018] systemd[1]: Finished register-nix-paths.service.882vm-test-run-nixbot-gitlab> gitlab # [    8.808426] systemd[1]: Reached target System Initialization.883vm-test-run-nixbot-gitlab> gitlab # [    8.809818] systemd[1]: Started Discard unused filesystem blocks once a week.884vm-test-run-nixbot-gitlab> gitlab # [    8.811189] systemd[1]: Started logrotate.timer.885vm-test-run-nixbot-gitlab> gitlab # [    8.812342] systemd[1]: Started Daily Cleanup of Temporary Directories.886vm-test-run-nixbot-gitlab> gitlab # [    8.813598] systemd[1]: Reached target Timer Units.887vm-test-run-nixbot-gitlab> gitlab # [    8.814363] systemd[1]: Listening on D-Bus System Message Bus Socket.888vm-test-run-nixbot-gitlab> gitlab # [    8.815302] systemd[1]: Listening on Nix Daemon Socket.889vm-test-run-nixbot-gitlab> gitlab # [    8.825965] systemd[1]: Starting nixbot CI service socket...890vm-test-run-nixbot-gitlab> gitlab # [    8.827601] systemd[1]: Listening on Hostname Service Socket.891vm-test-run-nixbot-gitlab> gitlab # [    8.830922] systemd[1]: Starting D-Bus System Message Bus...892vm-test-run-nixbot-gitlab> gitlab # [    8.832320] systemd[1]: Listening on nixbot CI service socket.893vm-test-run-nixbot-gitlab> gitlab # [    8.833259] systemd[1]: Reached target Socket Units.894vm-test-run-nixbot-gitlab> gitlab # [    8.872664] dbus-broker-launch[1040]: Looking up NSS user entry for 'systemd-timesync'...895vm-test-run-nixbot-gitlab> gitlab # [    8.874867] dbus-broker-launch[1040]: NSS returned no entry for 'systemd-timesync'896vm-test-run-nixbot-gitlab> gitlab # [    8.876131] dbus-broker-launch[1040]: Invalid user-name in /nix/store/ifm9k575wb3j66gvjx1xkdmmknzqr8v8-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"897vm-test-run-nixbot-gitlab> gitlab # [    8.878876] systemd[1]: Started D-Bus System Message Bus.898vm-test-run-nixbot-gitlab> gitlab # [    8.893209] dbus-broker-launch[1040]: Ready899vm-test-run-nixbot-gitlab> gitlab # [    8.894374] systemd[1]: Reached target Basic System.900vm-test-run-nixbot-gitlab> gitlab # [    8.896949] systemd[1]: Started backdoor.service.901vm-test-run-nixbot-gitlab> gitlab # [    8.903343] systemd[1]: Starting gitlab-config.service...902vm-test-run-nixbot-gitlab> gitlab # [    8.916393] systemd[1]: Starting Import lastlog data into lastlog2 database...903vm-test-run-nixbot-gitlab> gitlab # [    8.927398] systemd[1]: Starting Logrotate configuration check...904vm-test-run-nixbot-gitlab> gitlab # [    8.934207] systemd[1]: Starting Name Service Cache Daemon (nsncd)...905vm-test-run-nixbot-gitlab> gitlab # connecting to host...906vm-test-run-nixbot-gitlab> gitlab # [    8.939142] systemd[1]: Starting Post-Boot Actions...907vm-test-run-nixbot-gitlab> gitlab # [    8.940308] logrotate[1045]: warning: logrotate in debug mode does nothing except printing debug messages!  Consider using verbose mode (-v) instead if this is not what you want.908vm-test-run-nixbot-gitlab> gitlab # [    8.948093] logrotate[1045]: reading config file /nix/store/hk6ay5j24mh4h8mpr5rxlxxxp8v61fnj-logrotate.conf909vm-test-run-nixbot-gitlab> gitlab # [    8.949539] logrotate[1045]: note: 'monthly' overrides previously specified 'weekly'910vm-test-run-nixbot-gitlab> gitlab # [    8.950707] logrotate[1045]: Reading state from file: /var/lib/logrotate.status911vm-test-run-nixbot-gitlab> gitlab # [    8.951732] logrotate[1045]: state file /var/lib/logrotate.status does not exist912vm-test-run-nixbot-gitlab> gitlab # [    8.953025] logrotate[1045]: Allocating hash table for state file, size 64 entries913vm-test-run-nixbot-gitlab> gitlab: Guest shell says: b'Spawning backdoor root shell...\n'914vm-test-run-nixbot-gitlab> gitlab: connected to guest root shell915vm-test-run-nixbot-gitlab> gitlab: (connecting took 9.43 seconds)916vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for the VM to finish booting, in 9.43 seconds)917vm-test-run-nixbot-gitlab> gitlab # [    8.960114] logrotate[1045]: Handling 4 logs918vm-test-run-nixbot-gitlab> gitlab # [    8.963739] logrotate[1045]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed919vm-test-run-nixbot-gitlab> gitlab # [    8.966302] logrotate[1045]: considering log /var/log/btmp920vm-test-run-nixbot-gitlab> gitlab # [    8.967085] logrotate[1045]: Creating new state921vm-test-run-nixbot-gitlab> gitlab # [    8.967676] logrotate[1045]:   Now: 2026-06-29 11:56922vm-test-run-nixbot-gitlab> gitlab # [    8.968356] logrotate[1045]:   Last rotated at 2026-06-29 11:00923vm-test-run-nixbot-gitlab> gitlab # [    8.969149] logrotate[1045]:   log does not need rotating (log has already been rotated)924vm-test-run-nixbot-gitlab> gitlab # [    8.970145] logrotate[1045]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed925vm-test-run-nixbot-gitlab> gitlab # [    8.972257] logrotate[1045]: considering log /var/log/wtmp926vm-test-run-nixbot-gitlab> gitlab # [    8.972958] logrotate[1045]: Creating new state927vm-test-run-nixbot-gitlab> gitlab # [    8.973552] logrotate[1045]:   Now: 2026-06-29 11:56928vm-test-run-nixbot-gitlab> gitlab # [    8.974187] logrotate[1045]:   Last rotated at 2026-06-29 11:00929vm-test-run-nixbot-gitlab> gitlab # [    8.975018] logrotate[1045]:   log does not need rotating (log has already been rotated)930vm-test-run-nixbot-gitlab> gitlab # [    8.976128] logrotate[1045]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed931vm-test-run-nixbot-gitlab> gitlab # [    8.978013] logrotate[1045]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1045)932vm-test-run-nixbot-gitlab> gitlab # [    8.979115] logrotate[1045]: considering log /var/gitlab/state/log/*.log933vm-test-run-nixbot-gitlab> gitlab # [    8.980438] logrotate[1045]:   log /var/gitlab/state/log/*.log does not exist -- skipping934vm-test-run-nixbot-gitlab> gitlab # [    8.984603] logrotate[1045]: Creating new state935vm-test-run-nixbot-gitlab> gitlab # [    8.985736] logrotate[1045]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1045)936vm-test-run-nixbot-gitlab> gitlab # [    8.987185] logrotate[1045]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed937vm-test-run-nixbot-gitlab> gitlab # [    8.989590] logrotate[1045]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1045)938vm-test-run-nixbot-gitlab> gitlab # [    8.990722] logrotate[1045]: considering log /var/log/nginx/*.log939vm-test-run-nixbot-gitlab> gitlab # [    8.991567] logrotate[1045]:   log /var/log/nginx/*.log does not exist -- skipping940vm-test-run-nixbot-gitlab> gitlab # [    8.992679] logrotate[1045]: Creating new state941vm-test-run-nixbot-gitlab> gitlab # [    8.993374] logrotate[1045]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1045)942vm-test-run-nixbot-gitlab> gitlab # [    8.994478] systemd[1]: Started Reset console on configuration changes.943vm-test-run-nixbot-gitlab> gitlab # [    8.996587] nsncd[1049]: Jun 29 11:56:38.847 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"944vm-test-run-nixbot-gitlab> gitlab # [    8.999070] systemd[1]: Starting resolvconf update...945vm-test-run-nixbot-gitlab> gitlab # [    8.999935] systemd[1]: Started Name Service Cache Daemon (nsncd).946vm-test-run-nixbot-gitlab> gitlab # [    9.001691] systemd[1]: Finished Logrotate configuration check.947vm-test-run-nixbot-gitlab> gitlab # [    9.002748] systemd[1]: Finished Post-Boot Actions.948vm-test-run-nixbot-gitlab> gitlab # [    9.003538] systemd[1]: Reached target Host and Network Name Lookups.949vm-test-run-nixbot-gitlab> gitlab # [    9.004504] systemd[1]: Reached target User and Group Name Lookups.950vm-test-run-nixbot-gitlab> gitlab # [    9.008592] systemd[1]: Starting User Login Management...951vm-test-run-nixbot-gitlab> gitlab # [    9.014844] systemd[1]: Finished Import lastlog data into lastlog2 database.952vm-test-run-nixbot-gitlab> gitlab # [    9.068372] systemd-logind[1083]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard)953vm-test-run-nixbot-gitlab> gitlab # [    9.070946] systemd-logind[1083]: Watching system buttons on /dev/input/event0 (gpio-keys)954vm-test-run-nixbot-gitlab> gitlab # [    9.072356] systemd-logind[1083]: New seat seat0.955vm-test-run-nixbot-gitlab> gitlab # [    9.153508] systemd[1]: Stopped target Host and Network Name Lookups.956vm-test-run-nixbot-gitlab> gitlab # [    9.154604] systemd[1]: Stopping Host and Network Name Lookups...957vm-test-run-nixbot-gitlab> gitlab # [    9.155529] systemd[1]: Stopped target User and Group Name Lookups.958vm-test-run-nixbot-gitlab> gitlab # [    9.156465] systemd[1]: Stopping User and Group Name Lookups...959vm-test-run-nixbot-gitlab> gitlab # [    9.162923] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...960vm-test-run-nixbot-gitlab> gitlab # [    9.164057] systemd[1]: nscd.service: Deactivated successfully.961vm-test-run-nixbot-gitlab> gitlab # [    9.168691] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:08.0/virtio7/input/input3962vm-test-run-nixbot-gitlab> gitlab # [    9.166048] systemd[1]: Stopped Name Service Cache Daemon (nsncd).963vm-test-run-nixbot-gitlab> gitlab # [    9.168716] systemd[1]: nscd.service: Consumed 27ms CPU time over 199ms wall clock time, 8.1M memory peak, 5.9M read from disk.964vm-test-run-nixbot-gitlab> gitlab # [    9.170347] systemd[1]: Started User Login Management.965vm-test-run-nixbot-gitlab> gitlab # [    9.171503] systemd[1]: Starting linger-users.service...966vm-test-run-nixbot-gitlab> gitlab # [    9.197489] systemd[1]: Starting Name Service Cache Daemon (nsncd)...967vm-test-run-nixbot-gitlab> gitlab # [    9.200231] systemd[1]: linger-users.service: Deactivated successfully.968vm-test-run-nixbot-gitlab> gitlab # [    9.204256] systemd[1]: Finished linger-users.service.969vm-test-run-nixbot-gitlab> gitlab # [    9.209956] systemd[1]: Finished resolvconf update.970vm-test-run-nixbot-gitlab> gitlab # [    9.216354] nsncd[1143]: Jun 29 11:56:39.076 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"971vm-test-run-nixbot-gitlab> gitlab # [    9.218577] systemd[1]: Started Name Service Cache Daemon (nsncd).972vm-test-run-nixbot-gitlab> gitlab # [    9.219475] systemd[1]: Reached target Preparation for Network.973vm-test-run-nixbot-gitlab> gitlab # [    9.220870] systemd[1]: Reached target Host and Network Name Lookups.974vm-test-run-nixbot-gitlab> gitlab # [    9.221812] systemd[1]: Reached target User and Group Name Lookups.975vm-test-run-nixbot-gitlab> gitlab # [    9.226055] systemd[1]: Starting DHCP Client...976vm-test-run-nixbot-gitlab> gitlab # [    9.227230] systemd[1]: Starting Address configuration of eth1...977vm-test-run-nixbot-gitlab> gitlab # [    9.230644] systemd[1]: Starting Extra networking commands....978vm-test-run-nixbot-gitlab> gitlab # [    9.253095] systemd[1]: Starting Virtual Console Setup...979vm-test-run-nixbot-gitlab> gitlab # [    9.275010] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.980vm-test-run-nixbot-gitlab> gitlab # [    9.276243] systemd[1]: Stopped Virtual Console Setup.981vm-test-run-nixbot-gitlab> gitlab # [    9.279564] systemd[1]: Starting Virtual Console Setup...982vm-test-run-nixbot-gitlab> gitlab # [    9.305065] network-addresses-eth1-start[1160]: adding address 192.168.1.1/24... done983vm-test-run-nixbot-gitlab> gitlab # [    9.314678] network-addresses-eth1-start[1160]: adding address 2001:db8:1::1/64... done984vm-test-run-nixbot-gitlab> gitlab # [    9.328420] systemd[1]: Finished Address configuration of eth1.985vm-test-run-nixbot-gitlab> gitlab # [    9.346093] systemd-vconsole-setup[1173]: Configuration of first virtual console was skipped, ignoring remaining ones.986vm-test-run-nixbot-gitlab> gitlab # [    9.350780] systemd[1]: Finished Virtual Console Setup.987vm-test-run-nixbot-gitlab> gitlab # [    9.352790] systemd-logind[1083]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)988vm-test-run-nixbot-gitlab> gitlab # [    9.357462] systemd[1]: Finished Extra networking commands..989vm-test-run-nixbot-gitlab> gitlab # [    9.358957] systemd[1]: Reached target Network.990vm-test-run-nixbot-gitlab> gitlab # [    9.361268] systemd[1]: Starting gitlab-workhorse.service...991vm-test-run-nixbot-gitlab> gitlab # [    9.364447] dhcpcd[1175]: dhcpcd-10.3.2 starting992vm-test-run-nixbot-gitlab> gitlab # [    9.365199] systemd[1]: Starting Nginx Web Server...993vm-test-run-nixbot-gitlab> gitlab # [    9.368831] systemd[1]: Starting PostgreSQL Server...994vm-test-run-nixbot-gitlab> gitlab # [    9.372204] systemd[1]: Starting Redis Server - redis-gitlab...995vm-test-run-nixbot-gitlab> gitlab # [    9.375192] dhcpcd[1228]: dev: loaded udev996vm-test-run-nixbot-gitlab> gitlab # [    9.380135] systemd[1]: Starting Permit User Sessions...997vm-test-run-nixbot-gitlab> gitlab # [    9.389221] 8021q: 802.1Q VLAN Support v1.8998vm-test-run-nixbot-gitlab> gitlab # [    9.389610] 8021q: adding VLAN 0 to HW filter on device eth1999vm-test-run-nixbot-gitlab> gitlab # [    9.397770] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.1000vm-test-run-nixbot-gitlab> gitlab # [    9.411225] systemd[1]: Finished Permit User Sessions.1001vm-test-run-nixbot-gitlab> gitlab # [    9.420117] systemd[1]: Started Getty on tty1.1002vm-test-run-nixbot-gitlab> gitlab # [    9.420861] systemd[1]: Reached target Login Prompts.1003vm-test-run-nixbot-gitlab> gitlab # [    9.436199] cfg80211: Loading compiled-in X.509 certificates for regulatory database1004vm-test-run-nixbot-gitlab> gitlab # [    9.443482] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'1005vm-test-run-nixbot-gitlab> gitlab # [    9.443987] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'1006vm-test-run-nixbot-gitlab> gitlab # [    9.447351] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -21007vm-test-run-nixbot-gitlab> gitlab # [    9.447694] cfg80211: failed to load regulatory.db1008vm-test-run-nixbot-gitlab> gitlab # [    9.487351] 8021q: adding VLAN 0 to HW filter on device eth01009vm-test-run-nixbot-gitlab> gitlab # [    9.484294] dhcpcd[1228]: eth0: waiting for carrier1010vm-test-run-nixbot-gitlab> gitlab # [    9.487692] dhcpcd[1228]: eth0: carrier acquired1011vm-test-run-nixbot-gitlab> gitlab # [    9.494777] dhcpcd[1228]: DUID 00:01:00:01:31:d5:19:f7:52:54:00:12:34:561012vm-test-run-nixbot-gitlab> gitlab # [    9.495988] dhcpcd[1228]: eth0: IAID 00:12:34:561013vm-test-run-nixbot-gitlab> gitlab # [    9.496812] dhcpcd[1228]: eth0: adding address fe80::5054:ff:fe12:34561014vm-test-run-nixbot-gitlab> gitlab # [    9.550286] nginx-pre-start[1258]: nginx: the configuration file /nix/store/2526shpjv0bzachwf4qi968gxiiyv4kq-nginx.conf syntax is ok1015vm-test-run-nixbot-gitlab> gitlab # [    9.552384] nginx-pre-start[1258]: nginx: configuration file /nix/store/2526shpjv0bzachwf4qi968gxiiyv4kq-nginx.conf test is successful1016vm-test-run-nixbot-gitlab> gitlab # [    9.554604] redis-server[1252]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.1017vm-test-run-nixbot-gitlab> gitlab # [    9.557336] redis-server[1252]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1018vm-test-run-nixbot-gitlab> gitlab # [    9.558725] systemd[1]: Started Nginx Web Server.1019vm-test-run-nixbot-gitlab> gitlab # [    9.559478] redis-server[1252]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1252, just started1020vm-test-run-nixbot-gitlab> gitlab # [    9.561169] redis-server[1252]: Configuration loaded1021vm-test-run-nixbot-gitlab> gitlab # [    9.562006] redis[1252]: monotonic clock: ARM CNTVCT @ 1000 ticks/us1022vm-test-run-nixbot-gitlab> gitlab # [    9.563210] redis[1252]: Running mode=standalone, port=0.1023vm-test-run-nixbot-gitlab> gitlab # [    9.566286] redis[1252]: Server initialized1024vm-test-run-nixbot-gitlab> gitlab # [    9.567163] redis[1252]: Ready to accept connections unix1025vm-test-run-nixbot-gitlab> gitlab # [    9.568117] systemd[1]: Started Redis Server - redis-gitlab.1026vm-test-run-nixbot-gitlab> gitlab # [    9.569003] redis[1252]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1027vm-test-run-nixbot-gitlab> gitlab # [    9.572894] postgresql-pre-start[1256]: The files belonging to this database system will be owned by user "postgres".1028vm-test-run-nixbot-gitlab> gitlab # [    9.574609] postgresql-pre-start[1256]: This user must also own the server process.1029vm-test-run-nixbot-gitlab> gitlab # [    9.579551] postgresql-pre-start[1256]: The database cluster will be initialized with locale "en_US.UTF-8".1030vm-test-run-nixbot-gitlab> gitlab # [    9.581265] postgresql-pre-start[1256]: The default database encoding has accordingly been set to "UTF8".1031vm-test-run-nixbot-gitlab> gitlab # [    9.582857] postgresql-pre-start[1256]: The default text search configuration will be set to "english".1032vm-test-run-nixbot-gitlab> gitlab # [    9.584324] postgresql-pre-start[1256]: Data page checksums are disabled.1033vm-test-run-nixbot-gitlab> gitlab # [    9.585439] postgresql-pre-start[1256]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok1034vm-test-run-nixbot-gitlab> gitlab # [    9.587156] postgresql-pre-start[1256]: creating subdirectories ... ok1035vm-test-run-nixbot-gitlab> gitlab # [    9.588133] postgresql-pre-start[1256]: selecting dynamic shared memory implementation ... posix1036vm-test-run-nixbot-gitlab> gitlab # [    9.619078] postgresql-pre-start[1256]: selecting default "max_connections" ... 1001037vm-test-run-nixbot-gitlab> gitlab # [    9.658516] postgresql-pre-start[1256]: selecting default "shared_buffers" ... 128MB1038vm-test-run-nixbot-gitlab> gitlab # [    9.847435] systemd[1]: Started gitlab-workhorse.service.1039vm-test-run-nixbot-gitlab> gitlab # [   10.101183] postgresql-pre-start[1256]: selecting default time zone ... UTC1040vm-test-run-nixbot-gitlab> gitlab # [   10.102916] postgresql-pre-start[1256]: creating configuration files ... ok1041vm-test-run-nixbot-gitlab> gitlab # [   10.113532] dhcpcd[1228]: eth0: soliciting a DHCP lease1042vm-test-run-nixbot-gitlab> gitlab # [   10.139629] dhcpcd[1228]: eth0: offered 10.0.2.15 from 10.0.2.21043vm-test-run-nixbot-gitlab> gitlab # [   10.152562] dhcpcd[1228]: eth0: probing address 10.0.2.15/241044vm-test-run-nixbot-gitlab> gitlab # [   10.258853] gitlab-workhorse[1286]: time="2026-06-29T11:56:40Z" level=info msg=Starting build_time=19700101.000000 version=18.11.41045vm-test-run-nixbot-gitlab> gitlab # [   10.268166] gitlab-workhorse[1286]: time="2026-06-29T11:56:40Z" level=info msg="Using redis/go-redis"1046vm-test-run-nixbot-gitlab> gitlab # [   10.269568] gitlab-workhorse[1286]: time="2026-06-29T11:56:40Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1047vm-test-run-nixbot-gitlab> gitlab # [   10.332681] postgresql-pre-start[1256]: running bootstrap script ... ok1048vm-test-run-nixbot-gitlab> gitlab # [   10.951830] postgresql-pre-start[1256]: performing post-bootstrap initialization ... ok1049vm-test-run-nixbot-gitlab> gitlab # [   11.276863] postgresql-pre-start[1256]: syncing data to disk ... ok1050vm-test-run-nixbot-gitlab> gitlab # [   11.279615] postgresql-pre-start[1256]: initdb: warning: enabling "trust" authentication for local connections1051vm-test-run-nixbot-gitlab> gitlab # [   11.283705] postgresql-pre-start[1256]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1052vm-test-run-nixbot-gitlab> gitlab # [   11.289990] postgresql-pre-start[1256]: Success. You can now start the database server using:1053vm-test-run-nixbot-gitlab> gitlab # [   11.293537] postgresql-pre-start[1256]:     pg_ctl -D /var/lib/postgresql/17 -l logfile start1054vm-test-run-nixbot-gitlab> gitlab # [   11.494482] postgres[1320]: [1320] LOG:  starting PostgreSQL 17.10 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit1055vm-test-run-nixbot-gitlab> gitlab # [   11.500554] postgres[1320]: [1320] LOG:  listening on IPv6 address "::1", port 54321056vm-test-run-nixbot-gitlab> gitlab # [   11.504053] postgres[1320]: [1320] LOG:  listening on IPv4 address "127.0.0.1", port 54321057vm-test-run-nixbot-gitlab> gitlab # [   11.507527] postgres[1320]: [1320] LOG:  listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1058vm-test-run-nixbot-gitlab> gitlab # [   11.513743] postgres[1326]: [1326] LOG:  database system was shut down at 2026-06-29 11:56:40 GMT1059vm-test-run-nixbot-gitlab> gitlab # [   11.532304] postgres[1320]: [1320] LOG:  database system is ready to accept connections1060vm-test-run-nixbot-gitlab> gitlab # [   11.539740] systemd[1]: Started PostgreSQL Server.1061vm-test-run-nixbot-gitlab> gitlab # [   11.544948] systemd[1]: Starting PostgreSQL Setup Scripts...1062vm-test-run-nixbot-gitlab> gitlab # [   11.697102] dhcpcd[1228]: eth0: soliciting an IPv6 router1063vm-test-run-nixbot-gitlab> gitlab # [   11.699535] dhcpcd[1228]: eth0: Router Advertisement from fe80::21064vm-test-run-nixbot-gitlab> gitlab # [   11.702482] dhcpcd[1228]: eth0: adding address fec0::5054:ff:fe12:3456/641065vm-test-run-nixbot-gitlab> gitlab # [   11.705535] dhcpcd[1228]: eth0: adding route to fec0::/641066vm-test-run-nixbot-gitlab> gitlab # [   11.708155] dhcpcd[1228]: eth0: adding default route via fe80::21067vm-test-run-nixbot-gitlab> gitlab # [   11.803043] postgresql-setup-start[1342]: CREATE DATABASE1068vm-test-run-nixbot-gitlab> gitlab # [   11.833872] c0sj6d459nhf3wg9pm6ziy71l4581qq8-gitlab-config[1334]: mkdir -p /var/gitlab/state/home/.ssh: OK1069vm-test-run-nixbot-gitlab> gitlab # [   11.841940] c0sj6d459nhf3wg9pm6ziy71l4581qq8-gitlab-config[1334]: chmod 700 /var/gitlab/state/home/.ssh: OK1070vm-test-run-nixbot-gitlab> gitlab # [   11.851126] postgresql-setup-start[1348]: CREATE ROLE1071vm-test-run-nixbot-gitlab> gitlab # [   11.876304] postgresql-setup-start[1353]: ALTER DATABASE1072vm-test-run-nixbot-gitlab> gitlab # [   11.926344] postgresql-setup-start[1362]: CREATE ROLE1073vm-test-run-nixbot-gitlab> gitlab # [   11.930218] systemd[1]: Finished PostgreSQL Setup Scripts.1074vm-test-run-nixbot-gitlab> gitlab # [   11.932355] systemd[1]: Reached target PostgreSQL.1075vm-test-run-nixbot-gitlab> gitlab # [   11.936557] systemd[1]: Starting gitlab-postgresql.service...1076vm-test-run-nixbot-gitlab> gitlab # [   11.942013] systemd[1]: Finished gitlab-config.service.1077vm-test-run-nixbot-gitlab> gitlab # [   11.947572] systemd[1]: Started gitaly.service.1078vm-test-run-nixbot-gitlab> gitlab # [   11.952219] systemd[1]: Starting gitlab-db-config.service...1079vm-test-run-nixbot-gitlab> gitlab # [   12.126392] gitlab-postgresql-start[1378]: CREATE DATABASE1080vm-test-run-nixbot-gitlab> gitlab # [   12.217852] gitlab-postgresql-start[1383]: CREATE EXTENSION1081vm-test-run-nixbot-gitlab> gitlab # [   12.312603] gitlab-postgresql-start[1388]: CREATE EXTENSION1082vm-test-run-nixbot-gitlab> gitlab # [   12.317376] systemd[1]: Finished gitlab-postgresql.service.1083vm-test-run-nixbot-gitlab> gitlab # [   12.897113] gitaly[1370]: time="2026-06-29T11:56:42.760Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/krygc3bmd623znp2pddh3fv21q5f3nz2-gitaly.toml' and fix the errors that are printed." pid=13701084vm-test-run-nixbot-gitlab> gitlab # [   12.903158] gitaly[1370]: time="2026-06-29T11:56:42.761Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=13701085vm-test-run-nixbot-gitlab> gitlab # [   12.905879] gitaly[1370]: time="2026-06-29T11:56:42.761Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=13701086vm-test-run-nixbot-gitlab> gitlab # [   12.909350] gitaly[1370]: time="2026-06-29T11:56:42.761Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=13701087vm-test-run-nixbot-gitlab> gitlab # [   12.912356] gitaly[1370]: time="2026-06-29T11:56:42.761Z" level=info msg="Starting Gitaly" pid=1370 version=18.11.41088vm-test-run-nixbot-gitlab> gitlab # [   13.184458] gitaly[1370]: time="2026-06-29T11:56:43.048Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=285 pid=13701089vm-test-run-nixbot-gitlab> gitlab # [   13.187727] gitaly[1370]: time="2026-06-29T11:56:43.048Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=13701090vm-test-run-nixbot-gitlab> gitlab # [   13.189576] gitaly[1370]: time="2026-06-29T11:56:43.049Z" level=info msg="finished initializing command factory" duration_ms=1 pid=13701091vm-test-run-nixbot-gitlab> gitlab # [   13.191340] gitaly[1370]: time="2026-06-29T11:56:43.050Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1370/git-exec-2419547721.d/git pid=13701092vm-test-run-nixbot-gitlab> gitlab # [   13.194177] gitaly[1370]: time="2026-06-29T11:56:43.055Z" level=info msg="finished detecting git version" duration_ms=5 pid=13701093vm-test-run-nixbot-gitlab> gitlab # [   13.195963] gitaly[1370]: time="2026-06-29T11:56:43.055Z" level=info msg="using Git version" pid=1370 version=2.53.g1094vm-test-run-nixbot-gitlab> gitlab # [   13.197449] gitaly[1370]: time="2026-06-29T11:56:43.056Z" level=info msg="clearing disk cache object folder" pid=1370 storage=default1095vm-test-run-nixbot-gitlab> gitlab # [   13.200155] gitaly[1370]: time="2026-06-29T11:56:43.057Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2063849656 pid=1370 storage=default1096vm-test-run-nixbot-gitlab> gitlab # [   13.202585] gitaly[1370]: time="2026-06-29T11:56:43.057Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2063849656 pid=1370 storage=default1097vm-test-run-nixbot-gitlab> gitlab # [   13.206276] gitaly[1370]: time="2026-06-29T11:56:43.057Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=13701098vm-test-run-nixbot-gitlab> gitlab # [   13.209989] gitaly[1370]: time="2026-06-29T11:56:43.057Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=13701099vm-test-run-nixbot-gitlab> gitlab # [   13.213135] gitaly[1370]: time="2026-06-29T11:56:43.057Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2063849656 clear_duration_ms=0 pid=1370 storage=default1100vm-test-run-nixbot-gitlab> gitlab # [   13.216932] gitaly[1370]: time="2026-06-29T11:56:43.074Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=13701101vm-test-run-nixbot-gitlab> gitlab # [   13.219921] gitaly[1370]: time="2026-06-29T11:56:43.074Z" level=info msg="counting repositories" pid=13701102vm-test-run-nixbot-gitlab> gitlab # [   13.221863] gitaly[1370]: time="2026-06-29T11:56:43.074Z" level=info msg="starting to count repositories" pid=1370 storage_path=/var/gitlab/state/repositories1103vm-test-run-nixbot-gitlab> gitlab # [   13.224465] gitaly[1370]: time="2026-06-29T11:56:43.075Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1370 storage_path=/var/gitlab/state/repositories1104vm-test-run-nixbot-gitlab> gitlab # [   13.227322] gitaly[1370]: time="2026-06-29T11:56:43.075Z" level=info msg="completed counting all repositories" counting_duration=0 pid=13701105vm-test-run-nixbot-gitlab> gitlab # [   13.229897] gitaly[1370]: time="2026-06-29T11:56:43.074Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1370 schema=unix1106vm-test-run-nixbot-gitlab> gitlab # [   13.232612] gitaly[1370]: time="2026-06-29T11:56:43.075Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1370/sock.d/intern pid=1370 schema=unix1107vm-test-run-nixbot-gitlab> gitlab # [   13.235032] gitaly[1370]: time="2026-06-29T11:56:43.074Z" level=info msg="finished tempdir cleaner walk" pid=1370 storage=default time_ms=01108vm-test-run-nixbot-gitlab> gitlab # [   13.237492] gitaly[1370]: time="2026-06-29T11:56:43.076Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=13701109vm-test-run-nixbot-gitlab> gitlab # [   13.239533] gitaly[1370]: time="2026-06-29T11:56:43.076Z" level=info msg="Started Gitaly" duration_ms=314 pid=13701110vm-test-run-nixbot-gitlab> gitlab # [   13.241481] gitaly[1370]: time="2026-06-29T11:56:43.077Z" level=info msg="maintenance: daily scheduled" pid=1370 scheduled="2026-06-29 12:00:00 +0000 UTC"1111vm-test-run-nixbot-gitlab> gitlab # [   13.300331] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1376]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1112vm-test-run-nixbot-gitlab> gitlab # [   13.307507] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1376]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1113vm-test-run-nixbot-gitlab> gitlab # [   13.315009] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1376]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1114vm-test-run-nixbot-gitlab> gitlab # [   14.718599] dhcpcd[1228]: eth0: leased 10.0.2.15 for 86400 seconds1115vm-test-run-nixbot-gitlab> gitlab # [   14.720070] dhcpcd[1228]: eth0: adding route to 10.0.2.0/241116vm-test-run-nixbot-gitlab> gitlab # [   14.723057] dhcpcd[1228]: eth0: adding default route via 10.0.2.21117vm-test-run-nixbot-gitlab> gitlab # [   14.860958] systemd[1]: Started DHCP Client.1118vm-test-run-nixbot-gitlab> gitlab # [   14.863456] systemd[1]: Reached target Network is Online.1119vm-test-run-nixbot-gitlab> gitlab # [   16.730522] gitaly[1370]: time="2026-06-29T11:56:46.593Z" level=info msg="License database preloaded" duration_ms=3831 pid=13701120vm-test-run-nixbot-gitlab> gitlab # [   55.125005] postgres[1833]: [1833] ERROR:  relation "feature_gates" does not exist at character 5231121vm-test-run-nixbot-gitlab> gitlab # [   55.125607] postgres[1833]: [1833] STATEMENT:  SELECT a.attname, format_type(a.atttypid, a.atttypmod),1122vm-test-run-nixbot-gitlab> gitlab # [   55.125731] postgres[1833]: 	       pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1123vm-test-run-nixbot-gitlab> gitlab # [   55.125850] postgres[1833]: 	       c.collname, col_description(a.attrelid, a.attnum) AS comment,1124vm-test-run-nixbot-gitlab> gitlab # [   55.125988] postgres[1833]: 	       attidentity AS identity,1125vm-test-run-nixbot-gitlab> gitlab # [   55.126105] postgres[1833]: 	       attgenerated as attgenerated1126vm-test-run-nixbot-gitlab> gitlab # [   55.126221] postgres[1833]: 	  FROM pg_attribute a1127vm-test-run-nixbot-gitlab> gitlab # [   55.126338] postgres[1833]: 	  LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1128vm-test-run-nixbot-gitlab> gitlab # [   55.126441] postgres[1833]: 	  LEFT JOIN pg_type t ON a.atttypid = t.oid1129vm-test-run-nixbot-gitlab> gitlab # [   55.126540] postgres[1833]: 	  LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1130vm-test-run-nixbot-gitlab> gitlab # [   55.126667] postgres[1833]: 	 WHERE a.attrelid = '"feature_gates"'::regclass1131vm-test-run-nixbot-gitlab> gitlab # [   55.126793] postgres[1833]: 	   AND a.attnum > 0 AND NOT a.attisdropped1132vm-test-run-nixbot-gitlab> gitlab # [   55.126910] postgres[1833]: 	 ORDER BY a.attnum1133vm-test-run-nixbot-gitlab> gitlab # [   55.127024] postgres[1833]:1134vm-test-run-nixbot-gitlab> gitlab # [   55.388697] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1376]: W, [2026-06-29T11:57:25.251649 #1376]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1135vm-test-run-nixbot-gitlab> gitlab # [   60.350154] gitlab-workhorse[1286]: time="2026-06-29T11:57:30Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1136vm-test-run-nixbot-gitlab> gitlab # [   63.197403] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1906]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE:  extension "btree_gist" already exists, skipping1137vm-test-run-nixbot-gitlab> gitlab # [   63.197829] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1906]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE:  extension "pg_trgm" already exists, skipping1138vm-test-run-nixbot-gitlab> gitlab # [   70.351543] gitlab-workhorse[1286]: time="2026-06-29T11:57:40Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1139vm-test-run-nixbot-gitlab> gitlab # [   80.358684] gitlab-workhorse[1286]: time="2026-06-29T11:57:50Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1140vm-test-run-nixbot-gitlab> gitlab # [   86.496249] postgres[2000]: [2000] ERROR:  relation "table_without_models" does not exist1141vm-test-run-nixbot-gitlab> gitlab # [   86.499833] postgres[2000]: [2000] STATEMENT:  SELECT pg_get_serial_sequence('table_without_models', 'id')1142vm-test-run-nixbot-gitlab> gitlab # [   90.365864] gitlab-workhorse[1286]: time="2026-06-29T11:58:00Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1143vm-test-run-nixbot-gitlab> gitlab # [  100.367354] gitlab-workhorse[1286]: time="2026-06-29T11:58:10Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1144vm-test-run-nixbot-gitlab> gitlab # [  110.377426] gitlab-workhorse[1286]: time="2026-06-29T11:58:20Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1145vm-test-run-nixbot-gitlab> gitlab # [  117.159520] systemd[1]: Finished gitlab-db-config.service.1146vm-test-run-nixbot-gitlab> gitlab # [  117.164990] systemd[1]: Started gitlab-sidekiq.service.1147vm-test-run-nixbot-gitlab> gitlab # [  117.171247] systemd[1]: Starting gitlab.service...1148vm-test-run-nixbot-gitlab> gitlab # [  117.655721] bundle[2366]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1149vm-test-run-nixbot-gitlab> gitlab # [  117.662365] bundle[2366]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1150vm-test-run-nixbot-gitlab> gitlab # [  117.668480] bundle[2366]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1151vm-test-run-nixbot-gitlab> gitlab # [  120.386209] gitlab-workhorse[1286]: time="2026-06-29T11:58:30Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1152vm-test-run-nixbot-gitlab> gitlab # [  122.223880] sidekiq-cluster[2365]: {"severity":"INFO","time":"2026-06-29T11:58:32.060Z","message":"Starting cluster with 1 processes"}1153vm-test-run-nixbot-gitlab> gitlab # [  122.266035] bundle[2366]: {"timestamp":"2026-06-29T11:58:32.101Z","pid":2366,"message":"* Enabling systemd notification integration"}1154vm-test-run-nixbot-gitlab> gitlab # [  122.271094] bundle[2366]: {"timestamp":"2026-06-29T11:58:32.129Z","pid":2366,"message":"Puma starting in cluster mode..."}1155vm-test-run-nixbot-gitlab> gitlab # [  122.275705] bundle[2366]: {"timestamp":"2026-06-29T11:58:32.129Z","pid":2366,"message":"* Puma version: 7.2.0 (\"On The Corner\")"}1156vm-test-run-nixbot-gitlab> gitlab # [  122.280574] bundle[2366]: {"timestamp":"2026-06-29T11:58:32.129Z","pid":2366,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1157vm-test-run-nixbot-gitlab> gitlab # [  122.286480] bundle[2366]: {"timestamp":"2026-06-29T11:58:32.129Z","pid":2366,"message":"*  Min threads: 0"}1158vm-test-run-nixbot-gitlab> gitlab # [  122.290560] bundle[2366]: {"timestamp":"2026-06-29T11:58:32.129Z","pid":2366,"message":"*  Max threads: 4"}1159vm-test-run-nixbot-gitlab> gitlab # [  122.294667] bundle[2366]: {"timestamp":"2026-06-29T11:58:32.129Z","pid":2366,"message":"*  Environment: production"}1160vm-test-run-nixbot-gitlab> gitlab # [  122.299002] bundle[2366]: {"timestamp":"2026-06-29T11:58:32.129Z","pid":2366,"message":"*   Master PID: 2366"}1161vm-test-run-nixbot-gitlab> gitlab # [  122.303117] bundle[2366]: {"timestamp":"2026-06-29T11:58:32.129Z","pid":2366,"message":"*      Workers: 2"}1162vm-test-run-nixbot-gitlab> gitlab # [  122.307155] bundle[2366]: {"timestamp":"2026-06-29T11:58:32.129Z","pid":2366,"message":"*     Restarts: (✔) hot (✖) phased (✖) refork"}1163vm-test-run-nixbot-gitlab> gitlab # [  122.312321] bundle[2366]: {"timestamp":"2026-06-29T11:58:32.129Z","pid":2366,"message":"* Preloading application"}1164vm-test-run-nixbot-gitlab> gitlab # [  130.389278] gitlab-workhorse[1286]: time="2026-06-29T11:58:40Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1165vm-test-run-nixbot-gitlab> gitlab # [  140.390234] gitlab-workhorse[1286]: time="2026-06-29T11:58:50Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1166vm-test-run-nixbot-gitlab> gitlab # [  150.402173] gitlab-workhorse[1286]: time="2026-06-29T11:59:00Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1167vm-test-run-nixbot-gitlab> gitlab # [  159.562522] bundle[2366]: W, [2026-06-29T11:59:09.425524 #2366]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1168vm-test-run-nixbot-gitlab> gitlab # [  160.410312] gitlab-workhorse[1286]: time="2026-06-29T11:59:10Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1169vm-test-run-nixbot-gitlab> gitlab # [  164.614995] sidekiq-cluster[2415]: 2026-06-29T11:59:14.478Z pid=2415 tid=tt3 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2415, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>451350528}1170vm-test-run-nixbot-gitlab> gitlab # [  164.913765] sidekiq-cluster[2415]: W, [2026-06-29T11:59:14.777760 #2415]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1171vm-test-run-nixbot-gitlab> gitlab # [  164.973068] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:14.837Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1172vm-test-run-nixbot-gitlab> gitlab # [  168.247231] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.110Z","message":"GitLab reliable fetch activated!"}1173vm-test-run-nixbot-gitlab> gitlab # [  168.247624] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.110Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1174vm-test-run-nixbot-gitlab> gitlab # [  168.295122] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.159Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1175vm-test-run-nixbot-gitlab> gitlab # [  168.302830] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.166Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1176vm-test-run-nixbot-gitlab> gitlab # [  168.309644] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.173Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1177vm-test-run-nixbot-gitlab> gitlab # [  168.317222] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.180Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1178vm-test-run-nixbot-gitlab> gitlab # [  168.324084] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.186Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1179vm-test-run-nixbot-gitlab> gitlab # [  168.330176] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.193Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1180vm-test-run-nixbot-gitlab> gitlab # [  168.335232] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.197Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1181vm-test-run-nixbot-gitlab> gitlab # [  168.339971] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.202Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1182vm-test-run-nixbot-gitlab> gitlab # [  168.344184] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.207Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"}1183vm-test-run-nixbot-gitlab> gitlab # [  168.348886] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.213Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1184vm-test-run-nixbot-gitlab> gitlab # [  168.353187] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.217Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1185vm-test-run-nixbot-gitlab> gitlab # [  168.358549] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.222Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1186vm-test-run-nixbot-gitlab> gitlab # [  168.362398] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.226Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1187vm-test-run-nixbot-gitlab> gitlab # [  168.366525] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.229Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1188vm-test-run-nixbot-gitlab> gitlab # [  168.370529] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.233Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1189vm-test-run-nixbot-gitlab> gitlab # [  168.374549] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.237Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"}1190vm-test-run-nixbot-gitlab> gitlab # [  168.378662] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.241Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1191vm-test-run-nixbot-gitlab> gitlab # [  168.382529] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.246Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1192vm-test-run-nixbot-gitlab> gitlab # [  168.386167] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.250Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1193vm-test-run-nixbot-gitlab> gitlab # [  168.389972] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.253Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1194vm-test-run-nixbot-gitlab> gitlab # [  168.393445] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.257Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1195vm-test-run-nixbot-gitlab> gitlab # [  168.397383] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.261Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1196vm-test-run-nixbot-gitlab> gitlab # [  168.401305] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.265Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1197vm-test-run-nixbot-gitlab> gitlab # [  168.404736] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.268Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1198vm-test-run-nixbot-gitlab> gitlab # [  168.407844] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.271Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1199vm-test-run-nixbot-gitlab> gitlab # [  168.411179] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.274Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1200vm-test-run-nixbot-gitlab> gitlab # [  168.414333] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.278Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1201vm-test-run-nixbot-gitlab> gitlab # [  168.417633] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.281Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1202vm-test-run-nixbot-gitlab> gitlab # [  168.421012] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.285Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1203vm-test-run-nixbot-gitlab> gitlab # [  168.424313] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.288Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1204vm-test-run-nixbot-gitlab> gitlab # [  168.427774] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.292Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1205vm-test-run-nixbot-gitlab> gitlab # [  168.430882] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.294Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1206vm-test-run-nixbot-gitlab> gitlab # [  168.434279] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.298Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1207vm-test-run-nixbot-gitlab> gitlab # [  168.437502] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.301Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1208vm-test-run-nixbot-gitlab> gitlab # [  168.440793] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.305Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1209vm-test-run-nixbot-gitlab> gitlab # [  168.443764] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.308Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1210vm-test-run-nixbot-gitlab> gitlab # [  168.446638] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.310Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1211vm-test-run-nixbot-gitlab> gitlab # [  168.450453] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.313Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1212vm-test-run-nixbot-gitlab> gitlab # [  168.453070] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.315Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1213vm-test-run-nixbot-gitlab> gitlab # [  168.455687] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.318Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1214vm-test-run-nixbot-gitlab> gitlab # [  168.458427] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.320Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1215vm-test-run-nixbot-gitlab> gitlab # [  168.461196] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.323Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1216vm-test-run-nixbot-gitlab> gitlab # [  168.463891] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.325Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1217vm-test-run-nixbot-gitlab> gitlab # [  168.466671] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.327Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1218vm-test-run-nixbot-gitlab> gitlab # [  168.469204] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.330Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1219vm-test-run-nixbot-gitlab> gitlab # [  168.471798] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.333Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1220vm-test-run-nixbot-gitlab> gitlab # [  168.474536] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.335Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1221vm-test-run-nixbot-gitlab> gitlab # [  168.477033] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.338Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1222vm-test-run-nixbot-gitlab> gitlab # [  168.479394] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.340Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1223vm-test-run-nixbot-gitlab> gitlab # [  168.481796] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.343Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1224vm-test-run-nixbot-gitlab> gitlab # [  168.484056] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.346Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1225vm-test-run-nixbot-gitlab> gitlab # [  168.486244] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.349Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1226vm-test-run-nixbot-gitlab> gitlab # [  168.533055] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.397Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1227vm-test-run-nixbot-gitlab> gitlab # [  168.548647] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.412Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1228vm-test-run-nixbot-gitlab> gitlab # [  168.554360] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.417Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1229vm-test-run-nixbot-gitlab> gitlab # [  168.559751] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.419Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1230vm-test-run-nixbot-gitlab> gitlab # [  168.565021] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.423Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1231vm-test-run-nixbot-gitlab> gitlab # [  168.569738] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.426Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1232vm-test-run-nixbot-gitlab> gitlab # [  168.574389] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.430Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1233vm-test-run-nixbot-gitlab> gitlab # [  168.578649] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.432Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1234vm-test-run-nixbot-gitlab> gitlab # [  168.582679] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.436Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1235vm-test-run-nixbot-gitlab> gitlab # [  168.586119] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.438Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1236vm-test-run-nixbot-gitlab> gitlab # [  168.589417] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.442Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1237vm-test-run-nixbot-gitlab> gitlab # [  168.592756] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.446Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1238vm-test-run-nixbot-gitlab> gitlab # [  168.596045] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.449Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1239vm-test-run-nixbot-gitlab> gitlab # [  168.599212] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.452Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1240vm-test-run-nixbot-gitlab> gitlab # [  168.602369] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.454Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1241vm-test-run-nixbot-gitlab> gitlab # [  168.605243] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.457Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1242vm-test-run-nixbot-gitlab> gitlab # [  168.608118] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.459Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1243vm-test-run-nixbot-gitlab> gitlab # [  168.610995] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.462Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1244vm-test-run-nixbot-gitlab> gitlab # [  168.613959] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.464Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1245vm-test-run-nixbot-gitlab> gitlab # [  168.616682] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.466Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1246vm-test-run-nixbot-gitlab> gitlab # [  168.619239] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.468Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1247vm-test-run-nixbot-gitlab> gitlab # [  168.621831] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.470Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1248vm-test-run-nixbot-gitlab> gitlab # [  168.624417] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.473Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1249vm-test-run-nixbot-gitlab> gitlab # [  168.626874] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.475Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1250vm-test-run-nixbot-gitlab> gitlab # [  168.629486] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.477Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1251vm-test-run-nixbot-gitlab> gitlab # [  168.631981] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.479Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1252vm-test-run-nixbot-gitlab> gitlab # [  168.634818] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.481Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1253vm-test-run-nixbot-gitlab> gitlab # [  168.637575] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.483Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1254vm-test-run-nixbot-gitlab> gitlab # [  168.639882] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.485Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1255vm-test-run-nixbot-gitlab> gitlab # [  168.642567] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.487Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1256vm-test-run-nixbot-gitlab> gitlab # [  168.645259] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.490Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1257vm-test-run-nixbot-gitlab> gitlab # [  168.647712] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.494Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1258vm-test-run-nixbot-gitlab> gitlab # [  168.651232] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.501Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1259vm-test-run-nixbot-gitlab> gitlab # [  168.653876] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.505Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1260vm-test-run-nixbot-gitlab> gitlab # [  168.656269] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.508Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1261vm-test-run-nixbot-gitlab> gitlab # [  168.658762] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.514Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1262vm-test-run-nixbot-gitlab> gitlab # [  168.661191] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.521Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1263vm-test-run-nixbot-gitlab> gitlab # [  168.663576] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.523Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1264vm-test-run-nixbot-gitlab> gitlab # [  168.666067] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.525Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1265vm-test-run-nixbot-gitlab> gitlab # [  168.668480] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.528Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1266vm-test-run-nixbot-gitlab> gitlab # [  168.670845] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.530Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1267vm-test-run-nixbot-gitlab> gitlab # [  168.673387] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.532Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1268vm-test-run-nixbot-gitlab> gitlab # [  168.675991] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.534Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1269vm-test-run-nixbot-gitlab> gitlab # [  168.678207] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.537Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1270vm-test-run-nixbot-gitlab> gitlab # [  168.680975] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.539Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1271vm-test-run-nixbot-gitlab> gitlab # [  168.683408] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.543Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1272vm-test-run-nixbot-gitlab> gitlab # [  168.685993] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.544Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1273vm-test-run-nixbot-gitlab> gitlab # [  168.695079] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.559Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1274vm-test-run-nixbot-gitlab> gitlab # [  168.699469] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.561Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1275vm-test-run-nixbot-gitlab> gitlab # [  168.704158] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.563Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1276vm-test-run-nixbot-gitlab> gitlab # [  168.709510] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.566Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1277vm-test-run-nixbot-gitlab> gitlab # [  168.713435] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.569Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1278vm-test-run-nixbot-gitlab> gitlab # [  168.717444] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.571Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1279vm-test-run-nixbot-gitlab> gitlab # [  168.721702] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.574Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1280vm-test-run-nixbot-gitlab> gitlab # [  168.725847] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.575Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1281vm-test-run-nixbot-gitlab> gitlab # [  168.729443] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.577Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1282vm-test-run-nixbot-gitlab> gitlab # [  168.733035] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.580Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1283vm-test-run-nixbot-gitlab> gitlab # [  168.736374] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:18.582Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1284vm-test-run-nixbot-gitlab> gitlab # [  170.422077] gitlab-workhorse[1286]: time="2026-06-29T11:59:20Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1285vm-test-run-nixbot-gitlab> gitlab # [  180.423064] gitlab-workhorse[1286]: time="2026-06-29T11:59:30Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1286vm-test-run-nixbot-gitlab> gitlab # [  190.442221] gitlab-workhorse[1286]: time="2026-06-29T11:59:40Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1287vm-test-run-nixbot-gitlab> gitlab # [  200.450131] gitlab-workhorse[1286]: time="2026-06-29T11:59:50Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1288vm-test-run-nixbot-gitlab> gitlab # [  206.498409] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:56.361Z","message":"Booted Rails 7.2.3 application in production environment"}1289vm-test-run-nixbot-gitlab> gitlab # [  206.504370] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:56.361Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1290vm-test-run-nixbot-gitlab> gitlab # [  206.504560] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:56.361Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1291vm-test-run-nixbot-gitlab> gitlab # [  206.504736] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:56.361Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1292vm-test-run-nixbot-gitlab> gitlab # [  206.504875] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:56.364Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1293vm-test-run-nixbot-gitlab> gitlab # [  208.815950] bundle[2366]: {"timestamp":"2026-06-29T11:59:58.678Z","pid":2366,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1294vm-test-run-nixbot-gitlab> gitlab # [  208.816356] bundle[2366]: {"timestamp":"2026-06-29T11:59:58.679Z","pid":2366,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1295vm-test-run-nixbot-gitlab> gitlab # [  208.816488] bundle[2366]: {"timestamp":"2026-06-29T11:59:58.679Z","pid":2366,"message":"! #\u003cThread:0x0000f9d12ae84838 /nix/store/09wikq0z4z0s8mkafl8f2b6yp0p78frd-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/09wikq0z4z0s8mkafl8f2b6yp0p78frd-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1296vm-test-run-nixbot-gitlab> gitlab # [  208.816662] bundle[2366]: {"timestamp":"2026-06-29T11:59:58.679Z","pid":2366,"message":"! #\u003cThread:0x0000f9d12b0aedc0 /nix/store/q2bwpwxril8z19gq9b1nnv72drlwhhg8-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/q2bwpwxril8z19gq9b1nnv72drlwhhg8-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1297vm-test-run-nixbot-gitlab> gitlab # [  208.816774] bundle[2366]: {"timestamp":"2026-06-29T11:59:58.679Z","pid":2366,"message":"Use Ctrl-C to stop"}1298vm-test-run-nixbot-gitlab> gitlab # [  208.816890] bundle[2366]: {"timestamp":"2026-06-29T11:59:58.679Z","pid":2366,"message":"Sending status to systemd every 1.0 sec"}1299vm-test-run-nixbot-gitlab> gitlab # [  209.102742] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:58.966Z","message":"Cleaning working queues"}1300vm-test-run-nixbot-gitlab> gitlab # [  209.107852] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:58.970Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"2545588a9d80a59c77e0e5ec","created_at":"2026-06-29T11:58:25.895Z","trace_propagation_headers":{"sentry-trace":"78f70552938647468935529499a80f82-61c1a0a9d40e427c","baggage":"sentry-trace_id=78f70552938647468935529499a80f82,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"e2fe6fc48e4fc911456a91b4158d3c0f","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T11:58:25.907Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-2545588a9d80a59c77e0e5ec: start","job_status":"start","queue_duration_s":93.063272,"scheduling_latency_s":93.06334}1301vm-test-run-nixbot-gitlab> gitlab # [  209.452466] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T11:59:59.315Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"2545588a9d80a59c77e0e5ec","created_at":"2026-06-29T11:58:25.895Z","trace_propagation_headers":{"sentry-trace":"78f70552938647468935529499a80f82-61c1a0a9d40e427c","baggage":"sentry-trace_id=78f70552938647468935529499a80f82,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"e2fe6fc48e4fc911456a91b4158d3c0f","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T11:58:25.907Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-2545588a9d80a59c77e0e5ec: done: 0.345022 sec","job_status":"done","queue_duration_s":93.063272,"scheduling_latency_s":93.06334,"redis_calls":26,"redis_duration_s":0.015016000000000002,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.003802,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00892,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002294,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.178027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.19355412,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.345022,"completed_at":"2026-06-29T11:59:59.315Z","load_balancing_strategy":"primary","db_duration_s":0.10098,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1302vm-test-run-nixbot-gitlab> gitlab # [  209.907378] bundle[2366]: {"timestamp":"2026-06-29T11:59:59.771Z","pid":2366,"message":"- Worker 1 (PID: 3192) booted in 0.98s, phase: 0"}1303vm-test-run-nixbot-gitlab> gitlab # [  209.910535] bundle[2366]: {"timestamp":"2026-06-29T11:59:59.773Z","pid":2366,"message":"- Worker 0 (PID: 3188) booted in 1.01s, phase: 0"}1304vm-test-run-nixbot-gitlab> gitlab # [  209.914037] systemd[1]: Started gitlab.service.1305vm-test-run-nixbot-gitlab> gitlab # [  209.914787] systemd[1]: Reached target Common target for all GitLab services..1306vm-test-run-nixbot-gitlab> gitlab # [  209.919921] systemd[1]: Starting setup-gitlab.service...1307vm-test-run-nixbot-gitlab> gitlab # [  209.967502] setup-gitlab-start[3232]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1308vm-test-run-nixbot-gitlab> gitlab # [  210.137792] gitaly[1370]: time="2026-06-29T12:00:00.000Z" level=info msg="maintenance: daily starting" max_duration=600000000000 pid=13701309vm-test-run-nixbot-gitlab> gitlab # [  210.142948] gitaly[1370]: time="2026-06-29T12:00:00.001Z" level=info msg="maintenance: optimizing repos in storage" pid=1370 storage_path=/var/gitlab/state/repositories1310vm-test-run-nixbot-gitlab> gitlab # [  210.148800] gitaly[1370]: time="2026-06-29T12:00:00.001Z" level=info msg="maintenance: daily completed" actual_duration=1.70258ms error="<nil>" max_duration=600000000000 pid=13701311vm-test-run-nixbot-gitlab> gitlab # [  210.154601] gitaly[1370]: time="2026-06-29T12:00:00.001Z" level=info msg="maintenance: daily scheduled" pid=1370 scheduled="2026-06-30 12:00:00 +0000 UTC"1312vm-test-run-nixbot-gitlab> gitlab # [  210.188469] systemd[1]: Starting Logrotate Service...1313vm-test-run-nixbot-gitlab> gitlab # [  210.319602] systemd[1]: logrotate.service: Deactivated successfully.1314vm-test-run-nixbot-gitlab> gitlab # [  210.322685] systemd[1]: Finished Logrotate Service.1315vm-test-run-nixbot-gitlab> gitlab # [  213.211323] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:00:03 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 32011316vm-test-run-nixbot-gitlab> gitlab # [  213.219203] setup-gitlab-start[3264]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token1317vm-test-run-nixbot-gitlab> gitlab # [  213.224490] setup-gitlab-start[3265]: ++ jq -r .access_token1318vm-test-run-nixbot-gitlab> gitlab # [  214.255099] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:00:04 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 10131319vm-test-run-nixbot-gitlab> gitlab # [  214.263133] setup-gitlab-start[3232]: + BEARER=7a18b8207f9a42599941d936e50d9d6ea8ff4a1607b9899a71bc42ece84d43e71320vm-test-run-nixbot-gitlab> gitlab # [  214.264585] setup-gitlab-start[3232]: + AUTH='Authorization: Bearer 7a18b8207f9a42599941d936e50d9d6ea8ff4a1607b9899a71bc42ece84d43e7'1321vm-test-run-nixbot-gitlab> gitlab # [  214.266149] setup-gitlab-start[3232]: + curl -fs -X PUT -H 'Authorization: Bearer 7a18b8207f9a42599941d936e50d9d6ea8ff4a1607b9899a71bc42ece84d43e7' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true'1322vm-test-run-nixbot-gitlab> gitlab # [  215.182518] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:00:05 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 8891323vm-test-run-nixbot-gitlab> gitlab # [  215.186308] setup-gitlab-start[3232]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear1324vm-test-run-nixbot-gitlab> gitlab # [  215.206080] runuser[3283]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1325vm-test-run-nixbot-gitlab> gitlab # [  215.733861] setup-gitlab-start[3284]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1326vm-test-run-nixbot-gitlab> gitlab # [  215.734193] setup-gitlab-start[3284]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1327vm-test-run-nixbot-gitlab> gitlab # [  215.735594] setup-gitlab-start[3284]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1328vm-test-run-nixbot-gitlab> gitlab # [  216.510374] setup-gitlab-start[3284]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1329vm-test-run-nixbot-gitlab> gitlab # [  216.510784] setup-gitlab-start[3284]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1330vm-test-run-nixbot-gitlab> gitlab # [  216.516481] setup-gitlab-start[3284]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1331vm-test-run-nixbot-gitlab> gitlab # [  223.085926] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:12.948Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"ede744413d2ad90bdcbd6786","created_at":"2026-06-29T12:00:12.845Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"955564c00efe18e892bce7d80bc5b4c0","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:12.944Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-ede744413d2ad90bdcbd6786: start","job_status":"start","queue_duration_s":0.003539,"scheduling_latency_s":0.003602}1332vm-test-run-nixbot-gitlab> gitlab # [  223.278577] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:13.141Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"ede744413d2ad90bdcbd6786","created_at":"2026-06-29T12:00:12.845Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"955564c00efe18e892bce7d80bc5b4c0","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:12.944Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-ede744413d2ad90bdcbd6786: done: 0.193661 sec","job_status":"done","queue_duration_s":0.003539,"scheduling_latency_s":0.003602,"redis_calls":11,"redis_duration_s":0.016372,"redis_read_bytes":9,"redis_write_bytes":1239,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002075,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.010628,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.003669,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":298,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00330486,"duration_s":0.193661,"completed_at":"2026-06-29T12:00:13.141Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020183,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1333vm-test-run-nixbot-gitlab> gitlab # [  223.367819] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:13.151Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"2348c256cea4d6cb954a6a9e","created_at":"2026-06-29T12:00:12.978Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba829b3f32f8d6fbc1f8fcc168975365","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T12:00:12.988Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-2348c256cea4d6cb954a6a9e: start","job_status":"start","queue_duration_s":0.163591,"scheduling_latency_s":0.163637}1334vm-test-run-nixbot-gitlab> gitlab # [  223.465402] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:13.328Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"2348c256cea4d6cb954a6a9e","created_at":"2026-06-29T12:00:12.978Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba829b3f32f8d6fbc1f8fcc168975365","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T12:00:12.988Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-2348c256cea4d6cb954a6a9e: done: 0.176421 sec","job_status":"done","queue_duration_s":0.163591,"scheduling_latency_s":0.163637,"redis_calls":10,"redis_duration_s":0.038234,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002203,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.013816,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.022215,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00369064,"duration_s":0.176421,"completed_at":"2026-06-29T12:00:13.328Z","load_balancing_strategy":"primary","db_duration_s":0.061119,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1335vm-test-run-nixbot-gitlab> gitlab # [  223.557431] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:13.345Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"e61d8f8c19beed08499b8005","created_at":"2026-06-29T12:00:13.016Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bdae89273cff909d9cb203b14c232a00","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.025Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-e61d8f8c19beed08499b8005: start","job_status":"start","queue_duration_s":0.320688,"scheduling_latency_s":0.320741}1336vm-test-run-nixbot-gitlab> gitlab # [  223.628446] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:13.492Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1337vm-test-run-nixbot-gitlab> gitlab # [  223.679087] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:13.542Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"e61d8f8c19beed08499b8005","created_at":"2026-06-29T12:00:13.016Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bdae89273cff909d9cb203b14c232a00","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.025Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-e61d8f8c19beed08499b8005: done: 0.196392 sec","job_status":"done","queue_duration_s":0.320688,"scheduling_latency_s":0.320741,"redis_calls":10,"redis_duration_s":0.062436000000000005,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00575,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.050611,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.006075,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01810168,"duration_s":0.196392,"completed_at":"2026-06-29T12:00:13.542Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010943,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1338vm-test-run-nixbot-gitlab> gitlab # [  223.771440] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:13.566Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"8e6528df10b32075e3d0679a","created_at":"2026-06-29T12:00:13.030Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0267eb735c02422f2a7d81a8a05d386e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.038Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-8e6528df10b32075e3d0679a: start","job_status":"start","queue_duration_s":0.5272,"scheduling_latency_s":0.52725}1339vm-test-run-nixbot-gitlab> gitlab # [  223.866730] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:13.729Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"8e6528df10b32075e3d0679a","created_at":"2026-06-29T12:00:13.030Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0267eb735c02422f2a7d81a8a05d386e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.038Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-8e6528df10b32075e3d0679a: done: 0.16378 sec","job_status":"done","queue_duration_s":0.5272,"scheduling_latency_s":0.52725,"redis_calls":12,"redis_duration_s":0.03072,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.022383,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006404,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001933,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026264,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038322,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.16378,"completed_at":"2026-06-29T12:00:13.729Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004747,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1340vm-test-run-nixbot-gitlab> gitlab # [  223.959653] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:13.756Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ImportExportProjectCleanupWorker","jid":"e30b6461c9fab31dfa25e8b1","created_at":"2026-06-29T12:00:13.065Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69426c0db3389db742aaafa255fb3f8b","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:c1ab68355154ed2a8f72791841b7337bdce81d68788119e527d7446bb987ef9a","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.076Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ImportExportProjectCleanupWorker JID-e30b6461c9fab31dfa25e8b1: start","job_status":"start","queue_duration_s":0.679306,"scheduling_latency_s":0.679337}1341vm-test-run-nixbot-gitlab> gitlab # [  223.977641] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:13.840Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ImportExportProjectCleanupWorker","jid":"e30b6461c9fab31dfa25e8b1","created_at":"2026-06-29T12:00:13.065Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69426c0db3389db742aaafa255fb3f8b","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:c1ab68355154ed2a8f72791841b7337bdce81d68788119e527d7446bb987ef9a","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.076Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ImportExportProjectCleanupWorker JID-e30b6461c9fab31dfa25e8b1: done: 0.084333 sec","job_status":"done","queue_duration_s":0.679306,"scheduling_latency_s":0.679337,"redis_calls":12,"redis_duration_s":0.009713,"redis_read_bytes":11,"redis_write_bytes":1420,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002841,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":638,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005118,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001754,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00202326,"duration_s":0.084333,"completed_at":"2026-06-29T12:00:13.840Z","load_balancing_strategy":"primary","db_duration_s":0.023501,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1342vm-test-run-nixbot-gitlab> gitlab # [  224.023070] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:13.843Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ExpireBuildArtifactsWorker","jid":"10addf307b2b9b19f6f39201","created_at":"2026-06-29T12:00:13.087Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62c9c014f303774af74aada4eaee042d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:7be9ea93da0bda6381b334cf2932692b41842a57ea0aeac11899369225b9bb7e","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.099Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ExpireBuildArtifactsWorker JID-10addf307b2b9b19f6f39201: start","job_status":"start","queue_duration_s":0.744156,"scheduling_latency_s":0.744188}1343vm-test-run-nixbot-gitlab> gitlab # [  224.146670] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:14.008Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ExpireBuildArtifactsWorker","jid":"10addf307b2b9b19f6f39201","created_at":"2026-06-29T12:00:13.087Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62c9c014f303774af74aada4eaee042d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:7be9ea93da0bda6381b334cf2932692b41842a57ea0aeac11899369225b9bb7e","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.099Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ExpireBuildArtifactsWorker JID-10addf307b2b9b19f6f39201: done: 0.16495 sec","job_status":"done","queue_duration_s":0.744156,"scheduling_latency_s":0.744188,"redis_calls":10,"redis_duration_s":0.031252999999999996,"redis_read_bytes":9,"redis_write_bytes":1035,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.006984,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.022562,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001707,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031262,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00188214,"duration_s":0.16495,"completed_at":"2026-06-29T12:00:14.008Z","load_balancing_strategy":"primary","db_duration_s":0.002894,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1344vm-test-run-nixbot-gitlab> gitlab # [  224.244319] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:14.032Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"c20ffc13e2c6c4e3c3402ea5","created_at":"2026-06-29T12:00:13.120Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bc45723e4dbd8a2774a271128647660a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.148Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-c20ffc13e2c6c4e3c3402ea5: start","job_status":"start","queue_duration_s":0.883072,"scheduling_latency_s":0.883118}1345vm-test-run-nixbot-gitlab> gitlab # [  224.263429] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:14.101Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1346vm-test-run-nixbot-gitlab> gitlab # [  224.275215] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:14.138Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"c20ffc13e2c6c4e3c3402ea5","created_at":"2026-06-29T12:00:13.120Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bc45723e4dbd8a2774a271128647660a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.148Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-c20ffc13e2c6c4e3c3402ea5: done: 0.106374 sec","job_status":"done","queue_duration_s":0.883072,"scheduling_latency_s":0.883118,"redis_calls":10,"redis_duration_s":0.016798,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003456,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011486,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001856,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0006781,"duration_s":0.106374,"completed_at":"2026-06-29T12:00:14.138Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010827,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1347vm-test-run-nixbot-gitlab> gitlab # [  224.330500] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:14.144Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"0c3372b8bd6fdefd56f25eed","created_at":"2026-06-29T12:00:13.183Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"268648614355da78de41bcdf57257a27","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.206Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-0c3372b8bd6fdefd56f25eed: start","job_status":"start","queue_duration_s":0.93744,"scheduling_latency_s":0.937491}1348vm-test-run-nixbot-gitlab> gitlab # [  224.372944] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:14.235Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"0c3372b8bd6fdefd56f25eed","created_at":"2026-06-29T12:00:13.183Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"268648614355da78de41bcdf57257a27","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.206Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-0c3372b8bd6fdefd56f25eed: done: 0.09191 sec","job_status":"done","queue_duration_s":0.93744,"scheduling_latency_s":0.937491,"redis_calls":10,"redis_duration_s":0.003391,"redis_read_bytes":9,"redis_write_bytes":1097,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000704,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":309,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002035,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000652,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":218,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00053464,"duration_s":0.09191,"completed_at":"2026-06-29T12:00:14.235Z","load_balancing_strategy":"primary","db_duration_s":0.019486,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1349vm-test-run-nixbot-gitlab> gitlab # [  224.431970] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:14.246Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckCiJobsWorker","jid":"755d1c7bc1817bae8a459d33","created_at":"2026-06-29T12:00:13.266Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5823e5dd2b5c0a0c4d45aa96097e329b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:688518068b75df5544ce76496372eb2176b68f67fc20ec19a768c0a774774cd9","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.280Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckCiJobsWorker JID-755d1c7bc1817bae8a459d33: start","job_status":"start","queue_duration_s":0.966103,"scheduling_latency_s":0.966152}1350vm-test-run-nixbot-gitlab> gitlab # [  224.664372] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:14.526Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckCiJobsWorker","jid":"755d1c7bc1817bae8a459d33","created_at":"2026-06-29T12:00:13.266Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5823e5dd2b5c0a0c4d45aa96097e329b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:688518068b75df5544ce76496372eb2176b68f67fc20ec19a768c0a774774cd9","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.280Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckCiJobsWorker JID-755d1c7bc1817bae8a459d33: done: 0.280264 sec","job_status":"done","queue_duration_s":0.966103,"scheduling_latency_s":0.966152,"redis_calls":23,"redis_duration_s":0.052071,"redis_read_bytes":22,"redis_write_bytes":4956,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.020642,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1679,"redis_queues_calls":3,"redis_queues_duration_s":0.005826,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2314,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.020503,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":502,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.0051,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":461,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.065,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.033,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.113091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.10174854,"duration_s":0.280264,"completed_at":"2026-06-29T12:00:14.526Z","load_balancing_strategy":"primary","db_duration_s":0.078596,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1351vm-test-run-nixbot-gitlab> gitlab # [  224.756434] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:14.537Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"cef1559d5cf11d010c72ac69","created_at":"2026-06-29T12:00:13.286Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0548932c4e4b78e2ede7f5d94827ab7c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.300Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-cef1559d5cf11d010c72ac69: start","job_status":"start","queue_duration_s":1.237172,"scheduling_latency_s":1.237222}1352vm-test-run-nixbot-gitlab> gitlab # [  224.787342] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:14.650Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"cef1559d5cf11d010c72ac69","created_at":"2026-06-29T12:00:13.286Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0548932c4e4b78e2ede7f5d94827ab7c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.300Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-cef1559d5cf11d010c72ac69: done: 0.113012 sec","job_status":"done","queue_duration_s":1.237172,"scheduling_latency_s":1.237222,"redis_calls":13,"redis_duration_s":0.012476,"redis_read_bytes":139,"redis_write_bytes":1552,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.002502,"redis_feature_flag_read_bytes":132,"redis_feature_flag_write_bytes":680,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005905,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004069,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00183884,"duration_s":0.113012,"completed_at":"2026-06-29T12:00:14.650Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006737,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1353vm-test-run-nixbot-gitlab> gitlab # [  224.860647] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:14.657Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"56e3b51f8d0ec71031bfe319","created_at":"2026-06-29T12:00:13.305Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0d1e96f6f6638bbcafbee321a75cfe42","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.325Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-56e3b51f8d0ec71031bfe319: start","job_status":"start","queue_duration_s":1.332542,"scheduling_latency_s":1.332595}1354vm-test-run-nixbot-gitlab> gitlab # [  225.441189] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:15.303Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"56e3b51f8d0ec71031bfe319","created_at":"2026-06-29T12:00:13.305Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0d1e96f6f6638bbcafbee321a75cfe42","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.325Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-56e3b51f8d0ec71031bfe319: done: 0.646083 sec","job_status":"done","queue_duration_s":1.332542,"scheduling_latency_s":1.332595,"redis_calls":14,"redis_duration_s":0.006812,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002844,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002061,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001907,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.062,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":125,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.062,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.543419,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00127676,"exclusive_lock_hold_duration_s":0.58135346,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.646083,"completed_at":"2026-06-29T12:00:15.303Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.098468,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355vm-test-run-nixbot-gitlab> gitlab # [  225.537224] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:15.305Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"739f4b2ef264e07b58f1df12","created_at":"2026-06-29T12:00:13.335Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47bee0e134797d2d3989f06515206bad","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.351Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-739f4b2ef264e07b58f1df12: start","job_status":"start","queue_duration_s":1.953883,"scheduling_latency_s":1.953903}1356vm-test-run-nixbot-gitlab> gitlab # [  225.554836] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:15.343Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"739f4b2ef264e07b58f1df12","created_at":"2026-06-29T12:00:13.335Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47bee0e134797d2d3989f06515206bad","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.351Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-739f4b2ef264e07b58f1df12: done: 0.038497 sec","job_status":"done","queue_duration_s":1.953883,"scheduling_latency_s":1.953903,"redis_calls":13,"redis_duration_s":0.003709,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000616,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001868,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001225,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02295,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01351584,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.038497,"completed_at":"2026-06-29T12:00:15.343Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006646,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1357vm-test-run-nixbot-gitlab> gitlab # [  225.600470] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:15.345Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"2d7ccfa44d42b7c2b3b15de1","created_at":"2026-06-29T12:00:13.357Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bbdb4e33266bf4d044c8e626cc096f5d","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.366Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-2d7ccfa44d42b7c2b3b15de1: start","job_status":"start","queue_duration_s":1.978904,"scheduling_latency_s":1.978923}1358vm-test-run-nixbot-gitlab> gitlab # [  225.614914] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:15.374Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"2d7ccfa44d42b7c2b3b15de1","created_at":"2026-06-29T12:00:13.357Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bbdb4e33266bf4d044c8e626cc096f5d","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.366Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-2d7ccfa44d42b7c2b3b15de1: done: 0.028584 sec","job_status":"done","queue_duration_s":1.978904,"scheduling_latency_s":1.978923,"redis_calls":10,"redis_duration_s":0.0025830000000000002,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000496,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001549,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000538,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014701,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00056724,"duration_s":0.028584,"completed_at":"2026-06-29T12:00:15.374Z","load_balancing_strategy":"primary","db_duration_s":0.005891,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1359vm-test-run-nixbot-gitlab> gitlab # [  225.656892] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:15.375Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"2d2bc94651c03e874777b3e7","created_at":"2026-06-29T12:00:13.371Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d280f6c3f00a3a208add07ca02d1e86","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.386Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-2d2bc94651c03e874777b3e7: start","job_status":"start","queue_duration_s":1.989103,"scheduling_latency_s":1.98912}1360vm-test-run-nixbot-gitlab> gitlab # [  225.671425] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:15.490Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"2d2bc94651c03e874777b3e7","created_at":"2026-06-29T12:00:13.371Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d280f6c3f00a3a208add07ca02d1e86","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.386Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-2d2bc94651c03e874777b3e7: done: 0.115247 sec","job_status":"done","queue_duration_s":1.989103,"scheduling_latency_s":1.98912,"redis_calls":14,"redis_duration_s":0.006437,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001129,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002587,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002721,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005686,"exclusive_lock_hold_duration_s":0.042246259999999994,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.115247,"completed_at":"2026-06-29T12:00:15.490Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016101,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1361vm-test-run-nixbot-gitlab> gitlab # [  225.716751] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:15.564Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"859fc8d6d7865f20298d0269","created_at":"2026-06-29T12:00:13.392Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"375507f918f7170f935ac0401def0d96","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.400Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-859fc8d6d7865f20298d0269: start","job_status":"start","queue_duration_s":2.163599,"scheduling_latency_s":2.163644}1362vm-test-run-nixbot-gitlab> gitlab # [  225.857473] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:15.720Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"859fc8d6d7865f20298d0269","created_at":"2026-06-29T12:00:13.392Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"375507f918f7170f935ac0401def0d96","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.400Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-859fc8d6d7865f20298d0269: done: 0.156012 sec","job_status":"done","queue_duration_s":2.163599,"scheduling_latency_s":2.163644,"redis_calls":10,"redis_duration_s":0.014089,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002497,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007844,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003748,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0316,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00237148,"duration_s":0.156012,"completed_at":"2026-06-29T12:00:15.719Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001758,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1363vm-test-run-nixbot-gitlab> gitlab # [  225.925060] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:15.726Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"788f7f838f0435c52748e697","created_at":"2026-06-29T12:00:13.407Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"83f33d19602cb566b40cc9845fd4e38f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.416Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-788f7f838f0435c52748e697: start","job_status":"start","queue_duration_s":2.31043,"scheduling_latency_s":2.310475}1364vm-test-run-nixbot-gitlab> gitlab # [  226.117583] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:15.980Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"788f7f838f0435c52748e697","created_at":"2026-06-29T12:00:13.407Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"83f33d19602cb566b40cc9845fd4e38f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.416Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-788f7f838f0435c52748e697: done: 0.253833 sec","job_status":"done","queue_duration_s":2.31043,"scheduling_latency_s":2.310475,"redis_calls":19,"redis_duration_s":0.040003000000000004,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.008008,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.003351,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.021715,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.006929,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050126,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00348338,"duration_s":0.253833,"completed_at":"2026-06-29T12:00:15.980Z","load_balancing_strategy":"primary","db_duration_s":0.008058,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1365vm-test-run-nixbot-gitlab> gitlab # [  226.198113] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:15.986Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"8e66deaec0d9895e5d91a175","created_at":"2026-06-29T12:00:13.420Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16d46e20edd6effe8a8bd4fe572ea054","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.438Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-8e66deaec0d9895e5d91a175: start","job_status":"start","queue_duration_s":2.547842,"scheduling_latency_s":2.547887}1366vm-test-run-nixbot-gitlab> gitlab # [  226.214610] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:16.075Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"8e66deaec0d9895e5d91a175","created_at":"2026-06-29T12:00:13.420Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16d46e20edd6effe8a8bd4fe572ea054","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.438Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-8e66deaec0d9895e5d91a175: done: 0.088763 sec","job_status":"done","queue_duration_s":2.547842,"scheduling_latency_s":2.547887,"redis_calls":10,"redis_duration_s":0.004154,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000784,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002577,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000793,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04303,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00063402,"duration_s":0.088763,"completed_at":"2026-06-29T12:00:16.075Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016127,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1367vm-test-run-nixbot-gitlab> gitlab # [  226.261294] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:16.082Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"5d99cb4151c84da4c2859f2f","created_at":"2026-06-29T12:00:13.445Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82397943cdc2d09c51bdf23b197bab80","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.458Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-5d99cb4151c84da4c2859f2f: start","job_status":"start","queue_duration_s":2.624422,"scheduling_latency_s":2.62447}1368vm-test-run-nixbot-gitlab> gitlab # [  226.309747] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:16.172Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"5d99cb4151c84da4c2859f2f","created_at":"2026-06-29T12:00:13.445Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82397943cdc2d09c51bdf23b197bab80","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.458Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-5d99cb4151c84da4c2859f2f: done: 0.089744 sec","job_status":"done","queue_duration_s":2.624422,"scheduling_latency_s":2.62447,"redis_calls":10,"redis_duration_s":0.014693999999999999,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00065,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01014,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003904,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00231056,"duration_s":0.089744,"completed_at":"2026-06-29T12:00:16.172Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009324,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1369vm-test-run-nixbot-gitlab> gitlab # [  226.367473] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:16.187Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"3da773592f314f7adcbc4ae2","created_at":"2026-06-29T12:00:13.465Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b3ce507a93547b6193ac07e9ac34b9f7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T12:00:13.478Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-3da773592f314f7adcbc4ae2: start","job_status":"start","queue_duration_s":2.709201,"scheduling_latency_s":2.70925}1370vm-test-run-nixbot-gitlab> gitlab # [  226.382731] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:16.246Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1371vm-test-run-nixbot-gitlab> gitlab # [  226.414055] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:16.277Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"3da773592f314f7adcbc4ae2","created_at":"2026-06-29T12:00:13.465Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b3ce507a93547b6193ac07e9ac34b9f7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T12:00:13.478Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-3da773592f314f7adcbc4ae2: done: 0.089502 sec","job_status":"done","queue_duration_s":2.709201,"scheduling_latency_s":2.70925,"redis_calls":10,"redis_duration_s":0.019809999999999998,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004441,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01326,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002109,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032373,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00080026,"duration_s":0.089502,"completed_at":"2026-06-29T12:00:16.277Z","load_balancing_strategy":"primary","db_duration_s":0.004642,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372vm-test-run-nixbot-gitlab> gitlab # [  226.471141] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:16.283Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PruneOldEventsWorker","jid":"fd2e29daca23fa17363c6a64","created_at":"2026-06-29T12:00:13.489Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1007db7a25a0b16f3a7c3143712e5ab1","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:aa7847a0e96d8dccea25d7c51fc9fba0c73a3fc691400b9310229c28e3c4fab3","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.506Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"PruneOldEventsWorker JID-fd2e29daca23fa17363c6a64: start","job_status":"start","queue_duration_s":2.77647,"scheduling_latency_s":2.776513}1373vm-test-run-nixbot-gitlab> gitlab # [  226.523248] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:16.386Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PruneOldEventsWorker","jid":"fd2e29daca23fa17363c6a64","created_at":"2026-06-29T12:00:13.489Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1007db7a25a0b16f3a7c3143712e5ab1","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:aa7847a0e96d8dccea25d7c51fc9fba0c73a3fc691400b9310229c28e3c4fab3","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.506Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"PruneOldEventsWorker JID-fd2e29daca23fa17363c6a64: done: 0.103087 sec","job_status":"done","queue_duration_s":2.77647,"scheduling_latency_s":2.776513,"redis_calls":10,"redis_duration_s":0.008188,"redis_read_bytes":9,"redis_write_bytes":987,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002229,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":283,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005073,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000886,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00065434,"duration_s":0.103087,"completed_at":"2026-06-29T12:00:16.386Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02547,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1374vm-test-run-nixbot-gitlab> gitlab # [  226.587254] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:16.390Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"c0b627569c1dd3376fbbe6ae","created_at":"2026-06-29T12:00:13.522Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.544Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-c0b627569c1dd3376fbbe6ae: start","job_status":"start","queue_duration_s":2.845246,"scheduling_latency_s":2.845291}1375vm-test-run-nixbot-gitlab> gitlab # [  227.175752] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:17.038Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"c0b627569c1dd3376fbbe6ae","created_at":"2026-06-29T12:00:13.522Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.544Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-c0b627569c1dd3376fbbe6ae: done: 0.648647 sec","job_status":"done","queue_duration_s":2.845246,"scheduling_latency_s":2.845291,"redis_calls":71,"redis_duration_s":0.159058,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.005482,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.003674,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.147676,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002226,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.278742,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00379146,"duration_s":0.648647,"completed_at":"2026-06-29T12:00:17.038Z","load_balancing_strategy":"primary","db_duration_s":0.022376,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1376vm-test-run-nixbot-gitlab> gitlab # [  227.196225] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:17.058Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"5d7e90588200fb43644b4fb5","created_at":"2026-06-29T12:00:13.549Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0d2e42f7c40485526dbce065a0c1011","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.560Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-5d7e90588200fb43644b4fb5: start","job_status":"start","queue_duration_s":3.497529,"scheduling_latency_s":3.497578}1377vm-test-run-nixbot-gitlab> gitlab # [  230.149264] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.011Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"5d7e90588200fb43644b4fb5","created_at":"2026-06-29T12:00:13.549Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0d2e42f7c40485526dbce065a0c1011","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.560Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-5d7e90588200fb43644b4fb5: done: 2.953882 sec","job_status":"done","queue_duration_s":3.497529,"scheduling_latency_s":3.497578,"redis_calls":574,"redis_duration_s":1.6557979999999999,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004953,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.016597,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":1.634248,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.01296,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001107,"duration_s":2.953882,"completed_at":"2026-06-29T12:00:20.011Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004093,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378vm-test-run-nixbot-gitlab> gitlab # [  230.269239] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.132Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"4dd4f9b1ace1eeb925bc16de","created_at":"2026-06-29T12:00:13.565Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84195bb6f3a83cad3f6c680955e24366","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.576Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-4dd4f9b1ace1eeb925bc16de: start","job_status":"start","queue_duration_s":6.555863,"scheduling_latency_s":6.555928}1379vm-test-run-nixbot-gitlab> gitlab # [  230.447042] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.309Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"4dd4f9b1ace1eeb925bc16de","created_at":"2026-06-29T12:00:13.565Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84195bb6f3a83cad3f6c680955e24366","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.576Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-4dd4f9b1ace1eeb925bc16de: done: 0.177427 sec","job_status":"done","queue_duration_s":6.555863,"scheduling_latency_s":6.555928,"redis_calls":12,"redis_duration_s":0.014887999999999998,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000673,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00968,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004535,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.045,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.060507,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00144928,"exclusive_lock_hold_duration_s":0.050315679999999995,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.177427,"completed_at":"2026-06-29T12:00:20.309Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.072446,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380vm-test-run-nixbot-gitlab> gitlab # [  230.550195] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.316Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BulkImports::StaleImportWorker","jid":"b7b67a53cc5415250854cc5d","created_at":"2026-06-29T12:00:13.588Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d6d05f3398ca4659e7403cf294ed2f7","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1a00ac23ed20191d6dc4c4907281ed3c2c9a9fd8b3894e6dbd87102304111dab","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.596Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"BulkImports::StaleImportWorker JID-b7b67a53cc5415250854cc5d: start","job_status":"start","queue_duration_s":6.719272,"scheduling_latency_s":6.719317}1381vm-test-run-nixbot-gitlab> gitlab # [  230.677081] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.540Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BulkImports::StaleImportWorker","jid":"b7b67a53cc5415250854cc5d","created_at":"2026-06-29T12:00:13.588Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d6d05f3398ca4659e7403cf294ed2f7","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1a00ac23ed20191d6dc4c4907281ed3c2c9a9fd8b3894e6dbd87102304111dab","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.596Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"BulkImports::StaleImportWorker JID-b7b67a53cc5415250854cc5d: done: 0.224287 sec","job_status":"done","queue_duration_s":6.719272,"scheduling_latency_s":6.719317,"redis_calls":10,"redis_duration_s":0.025918,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.006506,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.013399,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.006013,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053592,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00251224,"duration_s":0.224287,"completed_at":"2026-06-29T12:00:20.540Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.113239,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382vm-test-run-nixbot-gitlab> gitlab # [  230.829152] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.692Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f8e35cb42461df5146db7e03","created_at":"2026-06-29T12:00:13.602Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a39eeb5907be19feaf4f9604c0d1fd12","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.611Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f8e35cb42461df5146db7e03: start","job_status":"start","queue_duration_s":7.080638,"scheduling_latency_s":7.08069}1383vm-test-run-nixbot-gitlab> gitlab # [  230.871003] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.713Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1384vm-test-run-nixbot-gitlab> gitlab # [  230.879486] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.722Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f8e35cb42461df5146db7e03","created_at":"2026-06-29T12:00:13.602Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a39eeb5907be19feaf4f9604c0d1fd12","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.611Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f8e35cb42461df5146db7e03: done: 0.030118 sec","job_status":"done","queue_duration_s":7.080638,"scheduling_latency_s":7.08069,"redis_calls":10,"redis_duration_s":0.0013289999999999999,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000285,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000759,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000285,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025998,"duration_s":0.030118,"completed_at":"2026-06-29T12:00:20.722Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000709,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1385vm-test-run-nixbot-gitlab> gitlab # [  230.943200] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.723Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ea4badf4a6f153cc712683e8","created_at":"2026-06-29T12:00:13.615Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"763823ec015707188faaec2a5461b747","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.624Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ea4badf4a6f153cc712683e8: start","job_status":"start","queue_duration_s":7.099131,"scheduling_latency_s":7.09915}1386vm-test-run-nixbot-gitlab> gitlab # [  230.962393] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.737Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1387vm-test-run-nixbot-gitlab> gitlab # [  230.967359] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.745Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ea4badf4a6f153cc712683e8","created_at":"2026-06-29T12:00:13.615Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"763823ec015707188faaec2a5461b747","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.624Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ea4badf4a6f153cc712683e8: done: 0.022077 sec","job_status":"done","queue_duration_s":7.099131,"scheduling_latency_s":7.09915,"redis_calls":10,"redis_duration_s":0.001559,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000366,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00031,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028762,"duration_s":0.022077,"completed_at":"2026-06-29T12:00:20.745Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000626,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388vm-test-run-nixbot-gitlab> gitlab # [  231.012785] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.747Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"270d1f018ce47cce522639a5","created_at":"2026-06-29T12:00:13.628Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dfb4ec7002345c64c259b2b45a22d0c0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.635Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-270d1f018ce47cce522639a5: start","job_status":"start","queue_duration_s":7.11191,"scheduling_latency_s":7.11193}1389vm-test-run-nixbot-gitlab> gitlab # [  231.025915] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.777Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"270d1f018ce47cce522639a5","created_at":"2026-06-29T12:00:13.628Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dfb4ec7002345c64c259b2b45a22d0c0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.635Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-270d1f018ce47cce522639a5: done: 0.030414 sec","job_status":"done","queue_duration_s":7.11191,"scheduling_latency_s":7.11193,"redis_calls":11,"redis_duration_s":0.0015630000000000002,"redis_read_bytes":10,"redis_write_bytes":1022,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000277,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":283,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000465,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":225,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016816,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025568,"duration_s":0.030414,"completed_at":"2026-06-29T12:00:20.777Z","load_balancing_strategy":"primary","db_duration_s":0.004996,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1390vm-test-run-nixbot-gitlab> gitlab # [  231.067055] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.778Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"3202236de007771cc6d2eeb2","created_at":"2026-06-29T12:00:13.639Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d56c7c6236bb352d79e8ac2ac67c0de","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.677Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-3202236de007771cc6d2eeb2: start","job_status":"start","queue_duration_s":7.101499,"scheduling_latency_s":7.101518}1391vm-test-run-nixbot-gitlab> gitlab # [  231.081242] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.830Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"3202236de007771cc6d2eeb2","created_at":"2026-06-29T12:00:13.639Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d56c7c6236bb352d79e8ac2ac67c0de","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.677Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-3202236de007771cc6d2eeb2: done: 0.051345 sec","job_status":"done","queue_duration_s":7.101499,"scheduling_latency_s":7.101518,"redis_calls":17,"redis_duration_s":0.003932,"redis_read_bytes":15,"redis_write_bytes":2007,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001796,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1150,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001183,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00052,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000433,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033509,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00053108,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.051345,"completed_at":"2026-06-29T12:00:20.830Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002388,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1392vm-test-run-nixbot-gitlab> gitlab # [  231.126180] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.833Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"8846ec670ee5adac06690284","created_at":"2026-06-29T12:00:13.686Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc43998a27f4c0e3ed2e5b862183dbfc","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.700Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-8846ec670ee5adac06690284: start","job_status":"start","queue_duration_s":7.133206,"scheduling_latency_s":7.133256}1393vm-test-run-nixbot-gitlab> gitlab # [  231.140800] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.912Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"8846ec670ee5adac06690284","created_at":"2026-06-29T12:00:13.686Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc43998a27f4c0e3ed2e5b862183dbfc","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.700Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-8846ec670ee5adac06690284: done: 0.078917 sec","job_status":"done","queue_duration_s":7.133206,"scheduling_latency_s":7.133256,"redis_calls":20,"redis_duration_s":0.006432,"redis_read_bytes":23,"redis_write_bytes":4316,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.00197,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1065,"redis_queues_calls":4,"redis_queues_duration_s":0.000908,"redis_queues_read_bytes":6,"redis_queues_write_bytes":2021,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.002934,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":1004,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00062,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004956,"duration_s":0.078917,"completed_at":"2026-06-29T12:00:20.912Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00235,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1394vm-test-run-nixbot-gitlab> gitlab # [  231.185022] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.915Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Releases::ManageEvidenceWorker","jid":"0e5857cdc18d2b4877daa53a","created_at":"2026-06-29T12:00:13.706Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"446230c0e61aa5f6ebd480b45746bc0e","meta.root_caller_id":"Cronjob","meta.feature_category":"release_evidence","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ca33015ef91ab20408351751cf9ad5e1289a7d78257a0117231d08455e951d7f","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.740Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Releases::ManageEvidenceWorker JID-0e5857cdc18d2b4877daa53a: start","job_status":"start","queue_duration_s":7.175388,"scheduling_latency_s":7.175445}1395vm-test-run-nixbot-gitlab> gitlab # [  231.199049] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.969Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Releases::ManageEvidenceWorker","jid":"0e5857cdc18d2b4877daa53a","created_at":"2026-06-29T12:00:13.706Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"446230c0e61aa5f6ebd480b45746bc0e","meta.root_caller_id":"Cronjob","meta.feature_category":"release_evidence","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ca33015ef91ab20408351751cf9ad5e1289a7d78257a0117231d08455e951d7f","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.740Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Releases::ManageEvidenceWorker JID-0e5857cdc18d2b4877daa53a: done: 0.053601 sec","job_status":"done","queue_duration_s":7.175388,"scheduling_latency_s":7.175445,"redis_calls":10,"redis_duration_s":0.0033399999999999997,"redis_read_bytes":9,"redis_write_bytes":1055,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000534,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002119,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000687,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":206,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029735,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0006565,"duration_s":0.053601,"completed_at":"2026-06-29T12:00:20.968Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010149,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1396vm-test-run-nixbot-gitlab> gitlab # [  231.241930] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:20.971Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"dbff1d38b065a2890509f253","created_at":"2026-06-29T12:00:13.754Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50b2fb793212477eef1a0d71840e0b12","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.789Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-dbff1d38b065a2890509f253: start","job_status":"start","queue_duration_s":7.181872,"scheduling_latency_s":7.181898}1397vm-test-run-nixbot-gitlab> gitlab # [  231.256544] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.012Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"dbff1d38b065a2890509f253","created_at":"2026-06-29T12:00:13.754Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50b2fb793212477eef1a0d71840e0b12","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.789Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-dbff1d38b065a2890509f253: done: 0.041537 sec","job_status":"done","queue_duration_s":7.181872,"scheduling_latency_s":7.181898,"redis_calls":13,"redis_duration_s":0.00368,"redis_read_bytes":10,"redis_write_bytes":1343,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000568,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001629,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001483,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":368,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00042346,"extra.packages_cleanup_delete_orphaned_dependencies_worker.last_processed_packages_dependency_id":0,"extra.packages_cleanup_delete_orphaned_dependencies_worker.deleted_rows_count":0,"duration_s":0.041537,"completed_at":"2026-06-29T12:00:21.012Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005465,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398vm-test-run-nixbot-gitlab> gitlab # [  231.302173] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.014Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"c1b6040be1d3d9d412dfaf35","created_at":"2026-06-29T12:00:13.816Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"d84fbf84048140430488581a5a29cdde","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.837Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-c1b6040be1d3d9d412dfaf35: start","job_status":"start","queue_duration_s":7.176675,"scheduling_latency_s":7.176697}1399vm-test-run-nixbot-gitlab> gitlab # [  231.316597] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.041Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"c1b6040be1d3d9d412dfaf35","created_at":"2026-06-29T12:00:13.816Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"d84fbf84048140430488581a5a29cdde","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.837Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-c1b6040be1d3d9d412dfaf35: done: 0.027257 sec","job_status":"done","queue_duration_s":7.176675,"scheduling_latency_s":7.176697,"redis_calls":10,"redis_duration_s":0.002598,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000536,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001532,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00053,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003956,"duration_s":0.027257,"completed_at":"2026-06-29T12:00:21.041Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00067,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400vm-test-run-nixbot-gitlab> gitlab # [  231.360273] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.043Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"RepositoryArchiveCacheWorker","jid":"a5337a2d3345007a80223d17","created_at":"2026-06-29T12:00:13.849Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"83136ffd111b7830804b76f66e850ca5","meta.root_caller_id":"Cronjob","meta.feature_category":"source_code_management","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:e95d964ef170969342a5c9d45a52aad8532569d2d839f18acfd130ccc124369d","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.863Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"RepositoryArchiveCacheWorker JID-a5337a2d3345007a80223d17: start","job_status":"start","queue_duration_s":7.18019,"scheduling_latency_s":7.180209}1401vm-test-run-nixbot-gitlab> gitlab # [  231.373866] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.067Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"RepositoryArchiveCacheWorker","jid":"a5337a2d3345007a80223d17","created_at":"2026-06-29T12:00:13.849Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"83136ffd111b7830804b76f66e850ca5","meta.root_caller_id":"Cronjob","meta.feature_category":"source_code_management","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:e95d964ef170969342a5c9d45a52aad8532569d2d839f18acfd130ccc124369d","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.863Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"RepositoryArchiveCacheWorker JID-a5337a2d3345007a80223d17: done: 0.024297 sec","job_status":"done","queue_duration_s":7.18019,"scheduling_latency_s":7.180209,"redis_calls":12,"redis_duration_s":0.003385,"redis_read_bytes":11,"redis_write_bytes":1392,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00105,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":640,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001856,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":206,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014603,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034112,"duration_s":0.024297,"completed_at":"2026-06-29T12:00:21.067Z","load_balancing_strategy":"primary","db_duration_s":0.001163,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402vm-test-run-nixbot-gitlab> gitlab # [  231.415083] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.069Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"edad13d1ca817204b6aaf6d8","created_at":"2026-06-29T12:00:13.875Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5f114a76c8f8893e0a5998abca86b0a2","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.885Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-edad13d1ca817204b6aaf6d8: start","job_status":"start","queue_duration_s":7.184096,"scheduling_latency_s":7.184115}1403vm-test-run-nixbot-gitlab> gitlab # [  231.429824] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.096Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"edad13d1ca817204b6aaf6d8","created_at":"2026-06-29T12:00:13.875Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5f114a76c8f8893e0a5998abca86b0a2","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.885Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-edad13d1ca817204b6aaf6d8: done: 0.027228 sec","job_status":"done","queue_duration_s":7.184096,"scheduling_latency_s":7.184115,"redis_calls":12,"redis_duration_s":0.0033280000000000002,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000541,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001475,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001312,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00060054,"exclusive_lock_hold_duration_s":0.00717528,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.027228,"completed_at":"2026-06-29T12:00:21.096Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004639,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404vm-test-run-nixbot-gitlab> gitlab # [  231.473294] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.097Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"2d4803a5b073d1f5e54c6104","created_at":"2026-06-29T12:00:13.912Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1acaae103021005bfc4168789206e6d7","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.920Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-2d4803a5b073d1f5e54c6104: start","job_status":"start","queue_duration_s":7.177244,"scheduling_latency_s":7.17726}1405vm-test-run-nixbot-gitlab> gitlab # [  231.486681] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.146Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"2d4803a5b073d1f5e54c6104","created_at":"2026-06-29T12:00:13.912Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1acaae103021005bfc4168789206e6d7","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.920Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-2d4803a5b073d1f5e54c6104: done: 0.048694 sec","job_status":"done","queue_duration_s":7.177244,"scheduling_latency_s":7.17726,"redis_calls":10,"redis_duration_s":0.002677,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00048,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001626,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000571,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00049644,"extra.pages_deactivate_expired_deployments_cron_worker.deactivate_expired_pages_deployments":{"deactivated_deployments":0,"duration":0.00218932},"duration_s":0.048694,"completed_at":"2026-06-29T12:00:21.146Z","load_balancing_strategy":"primary","db_duration_s":0.019852,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406vm-test-run-nixbot-gitlab> gitlab # [  231.531225] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.148Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"2a3ed3bdadb98f16d2fe659c","created_at":"2026-06-29T12:00:13.924Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1167a48e9ffe1441c2b14ba1659a4354","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.946Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-2a3ed3bdadb98f16d2fe659c: start","job_status":"start","queue_duration_s":7.202648,"scheduling_latency_s":7.202668}1407vm-test-run-nixbot-gitlab> gitlab # [  231.545477] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.188Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"2a3ed3bdadb98f16d2fe659c","created_at":"2026-06-29T12:00:13.924Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1167a48e9ffe1441c2b14ba1659a4354","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.946Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-2a3ed3bdadb98f16d2fe659c: done: 0.039875 sec","job_status":"done","queue_duration_s":7.202648,"scheduling_latency_s":7.202668,"redis_calls":10,"redis_duration_s":0.001573,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000304,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000895,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000374,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021435,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002599,"duration_s":0.039875,"completed_at":"2026-06-29T12:00:21.188Z","load_balancing_strategy":"primary","db_duration_s":0.011932,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408vm-test-run-nixbot-gitlab> gitlab # [  231.587830] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.190Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"4bff801cb3850171e0e3d890","created_at":"2026-06-29T12:00:13.954Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d3e41e679b7442b6719e539f2c6f03e","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.979Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-4bff801cb3850171e0e3d890: start","job_status":"start","queue_duration_s":7.210297,"scheduling_latency_s":7.210315}1409vm-test-run-nixbot-gitlab> gitlab # [  231.602008] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.211Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"4bff801cb3850171e0e3d890","created_at":"2026-06-29T12:00:13.954Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d3e41e679b7442b6719e539f2c6f03e","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:13.979Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-4bff801cb3850171e0e3d890: done: 0.021922 sec","job_status":"done","queue_duration_s":7.210297,"scheduling_latency_s":7.210315,"redis_calls":10,"redis_duration_s":0.0018490000000000002,"redis_read_bytes":9,"redis_write_bytes":1179,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000275,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001244,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00033,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":238,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014587,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002529,"duration_s":0.021922,"completed_at":"2026-06-29T12:00:21.211Z","load_balancing_strategy":"primary","db_duration_s":0.000642,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410vm-test-run-nixbot-gitlab> gitlab # [  231.645056] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.213Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["[FILTERED]"],"class":"Database::BackgroundOperation::CronEnqueueWorker","jid":"91cf840c359d3c7371eefeb9","created_at":"2026-06-29T12:00:14.000Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d1911fd08943effd0f5294cf66a0c8d7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:38f97efb1ac6a87ded5913833f7d951478e91a6be2a4328abbe154ee188cf7fb","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.017Z","job_size_bytes":101,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CronEnqueueWorker JID-91cf840c359d3c7371eefeb9: start","job_status":"start","queue_duration_s":7.195434,"scheduling_latency_s":7.195452}1411vm-test-run-nixbot-gitlab> gitlab # [  231.659921] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.292Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["[FILTERED]"],"class":"Database::BackgroundOperation::CronEnqueueWorker","jid":"91cf840c359d3c7371eefeb9","created_at":"2026-06-29T12:00:14.000Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d1911fd08943effd0f5294cf66a0c8d7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:38f97efb1ac6a87ded5913833f7d951478e91a6be2a4328abbe154ee188cf7fb","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.017Z","job_size_bytes":101,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CronEnqueueWorker JID-91cf840c359d3c7371eefeb9: done: 0.078902 sec","job_status":"done","queue_duration_s":7.195434,"scheduling_latency_s":7.195452,"redis_calls":10,"redis_duration_s":0.001326,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000264,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000779,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000283,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051527,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024442,"duration_s":0.078902,"completed_at":"2026-06-29T12:00:21.291Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023092,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412vm-test-run-nixbot-gitlab> gitlab # [  231.703992] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.293Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"62465db4e7fc8446b5acd67f","created_at":"2026-06-29T12:00:14.030Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"181e09b1bda37380b8f2b1f54caec6ea","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.044Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-62465db4e7fc8446b5acd67f: start","job_status":"start","queue_duration_s":7.248176,"scheduling_latency_s":7.248194}1413vm-test-run-nixbot-gitlab> gitlab # [  231.718476] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.319Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"62465db4e7fc8446b5acd67f","created_at":"2026-06-29T12:00:14.030Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"181e09b1bda37380b8f2b1f54caec6ea","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.044Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-62465db4e7fc8446b5acd67f: done: 0.026565 sec","job_status":"done","queue_duration_s":7.248176,"scheduling_latency_s":7.248194,"redis_calls":10,"redis_duration_s":0.001322,"redis_read_bytes":9,"redis_write_bytes":1109,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000271,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000777,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000274,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013659,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025342,"duration_s":0.026565,"completed_at":"2026-06-29T12:00:21.319Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007359,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414vm-test-run-nixbot-gitlab> gitlab # [  231.762663] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.320Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"898338e81e259ca484e70cf5","created_at":"2026-06-29T12:00:14.075Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb251a18cd212f66bd305d5e24cc756f","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.093Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-898338e81e259ca484e70cf5: start","job_status":"start","queue_duration_s":7.227771,"scheduling_latency_s":7.227789}1415vm-test-run-nixbot-gitlab> gitlab # [  231.776141] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.338Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"898338e81e259ca484e70cf5","created_at":"2026-06-29T12:00:14.075Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb251a18cd212f66bd305d5e24cc756f","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.093Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-898338e81e259ca484e70cf5: done: 0.017909 sec","job_status":"done","queue_duration_s":7.227771,"scheduling_latency_s":7.227789,"redis_calls":12,"redis_duration_s":0.001637,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000357,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000265,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000727,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000288,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023088,"duration_s":0.017909,"completed_at":"2026-06-29T12:00:21.338Z","load_balancing_strategy":"primary","db_duration_s":0.000582,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416vm-test-run-nixbot-gitlab> gitlab # [  231.819734] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.339Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"d44528dd1566ee6b22a80b41","created_at":"2026-06-29T12:00:14.128Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37ca0923e2655c19cb2c7bf42cc726a1","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.147Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-d44528dd1566ee6b22a80b41: start","job_status":"start","queue_duration_s":7.192343,"scheduling_latency_s":7.19236}1417vm-test-run-nixbot-gitlab> gitlab # [  231.834976] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.362Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"d44528dd1566ee6b22a80b41","created_at":"2026-06-29T12:00:14.128Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37ca0923e2655c19cb2c7bf42cc726a1","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.147Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-d44528dd1566ee6b22a80b41: done: 0.023075 sec","job_status":"done","queue_duration_s":7.192343,"scheduling_latency_s":7.19236,"redis_calls":12,"redis_duration_s":0.0016359999999999999,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000263,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00073,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000643,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00043532,"exclusive_lock_hold_duration_s":0.00581866,"duration_s":0.023075,"completed_at":"2026-06-29T12:00:21.362Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003844,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418vm-test-run-nixbot-gitlab> gitlab # [  231.878912] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.364Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6543581473f97989b30b0484","created_at":"2026-06-29T12:00:14.194Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e448720e073391330fd5d2aeecfbe85","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T12:00:14.231Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6543581473f97989b30b0484: start","job_status":"start","queue_duration_s":7.132331,"scheduling_latency_s":7.132349}1419vm-test-run-nixbot-gitlab> gitlab # [  231.892637] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.374Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1420vm-test-run-nixbot-gitlab> gitlab # [  231.895851] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.381Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6543581473f97989b30b0484","created_at":"2026-06-29T12:00:14.194Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e448720e073391330fd5d2aeecfbe85","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T12:00:14.231Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6543581473f97989b30b0484: done: 0.01709 sec","job_status":"done","queue_duration_s":7.132331,"scheduling_latency_s":7.132349,"redis_calls":10,"redis_duration_s":0.0012439999999999999,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000265,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000704,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000275,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002326,"duration_s":0.01709,"completed_at":"2026-06-29T12:00:21.381Z","load_balancing_strategy":"primary","db_duration_s":0.000604,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421vm-test-run-nixbot-gitlab> gitlab # [  231.940467] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.382Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"b53fc5d644b5b4238dd034d9","created_at":"2026-06-29T12:00:14.248Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"87257cc9172a0ae64517b763ae7c2f2c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.263Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-b53fc5d644b5b4238dd034d9: start","job_status":"start","queue_duration_s":7.118619,"scheduling_latency_s":7.118636}1422vm-test-run-nixbot-gitlab> gitlab # [  231.955141] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.584Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"b53fc5d644b5b4238dd034d9","created_at":"2026-06-29T12:00:14.248Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"87257cc9172a0ae64517b763ae7c2f2c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.263Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-b53fc5d644b5b4238dd034d9: done: 0.201819 sec","job_status":"done","queue_duration_s":7.118619,"scheduling_latency_s":7.118636,"redis_calls":12,"redis_duration_s":0.001733,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000266,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000773,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000694,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.187227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00043306,"exclusive_lock_hold_duration_s":0.18447096,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.201819,"completed_at":"2026-06-29T12:00:21.584Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013166,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null}1423vm-test-run-nixbot-gitlab> gitlab # [  231.999888] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.585Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"3ad32b75f51cd3fbf14dd798","created_at":"2026-06-29T12:00:14.288Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b6e7e9c7b0e2cda293f5cc4089067690","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.319Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-3ad32b75f51cd3fbf14dd798: start","job_status":"start","queue_duration_s":7.265287,"scheduling_latency_s":7.265305}1424vm-test-run-nixbot-gitlab> gitlab # [  232.013322] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.606Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"3ad32b75f51cd3fbf14dd798","created_at":"2026-06-29T12:00:14.288Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b6e7e9c7b0e2cda293f5cc4089067690","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.319Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-3ad32b75f51cd3fbf14dd798: done: 0.021005 sec","job_status":"done","queue_duration_s":7.265287,"scheduling_latency_s":7.265305,"redis_calls":10,"redis_duration_s":0.001371,"redis_read_bytes":9,"redis_write_bytes":1087,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000289,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000291,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024216,"duration_s":0.021005,"completed_at":"2026-06-29T12:00:21.606Z","load_balancing_strategy":"primary","db_duration_s":0.000319,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425vm-test-run-nixbot-gitlab> gitlab # [  232.054680] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.607Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","jid":"f6924844defebbe6bc6ca810","created_at":"2026-06-29T12:00:14.363Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec92a78e13dcc629aef582875d7dbfe8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fd978bae1387dafc89025ea66dbcf41624e32b5e06562317011e023b1327155","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.384Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleBulkDeleteJobArtifactCronWorker JID-f6924844defebbe6bc6ca810: start","job_status":"start","queue_duration_s":7.222649,"scheduling_latency_s":7.222668}1426vm-test-run-nixbot-gitlab> gitlab # [  232.069448] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.640Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","jid":"f6924844defebbe6bc6ca810","created_at":"2026-06-29T12:00:14.363Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec92a78e13dcc629aef582875d7dbfe8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fd978bae1387dafc89025ea66dbcf41624e32b5e06562317011e023b1327155","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.384Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleBulkDeleteJobArtifactCronWorker JID-f6924844defebbe6bc6ca810: done: 0.033296 sec","job_status":"done","queue_duration_s":7.222649,"scheduling_latency_s":7.222668,"redis_calls":24,"redis_duration_s":0.0031219999999999998,"redis_read_bytes":25,"redis_write_bytes":5967,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000592,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":704,"redis_queues_calls":2,"redis_queues_duration_s":0.000228,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3800,"redis_queues_metadata_calls":11,"redis_queues_metadata_duration_s":0.00148,"redis_queues_metadata_read_bytes":14,"redis_queues_metadata_write_bytes":894,"redis_shared_state_calls":7,"redis_shared_state_duration_s":0.000822,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":569,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025426,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026632,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.max_buckets":5,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.available_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.occupied_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.stale_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.missing_count":5,"duration_s":0.033296,"completed_at":"2026-06-29T12:00:21.640Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000734,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427vm-test-run-nixbot-gitlab> gitlab # [  232.117832] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.641Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"18fe5515aeb451374fa30e35","created_at":"2026-06-29T12:00:14.403Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7189a3c45f9995ece7a4b17e14291ccb","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.415Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-18fe5515aeb451374fa30e35: start","job_status":"start","queue_duration_s":7.226641,"scheduling_latency_s":7.22666}1428vm-test-run-nixbot-gitlab> gitlab # [  232.131887] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.671Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"18fe5515aeb451374fa30e35","created_at":"2026-06-29T12:00:14.403Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7189a3c45f9995ece7a4b17e14291ccb","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.415Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-18fe5515aeb451374fa30e35: done: 0.029901 sec","job_status":"done","queue_duration_s":7.226641,"scheduling_latency_s":7.22666,"redis_calls":10,"redis_duration_s":0.0014529999999999999,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000296,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000856,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000301,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025658,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.029901,"completed_at":"2026-06-29T12:00:21.671Z","load_balancing_strategy":"primary","db_duration_s":0.00888,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429vm-test-run-nixbot-gitlab> gitlab # [  232.176280] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.672Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"571f2f7b1b769e821dbb384a","created_at":"2026-06-29T12:00:14.454Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.481Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-571f2f7b1b769e821dbb384a: start","job_status":"start","queue_duration_s":7.191258,"scheduling_latency_s":7.191276}1430vm-test-run-nixbot-gitlab> gitlab # [  232.191237] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.717Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"571f2f7b1b769e821dbb384a","created_at":"2026-06-29T12:00:14.454Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.481Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-571f2f7b1b769e821dbb384a: done: 0.044968 sec","job_status":"done","queue_duration_s":7.191258,"scheduling_latency_s":7.191276,"redis_calls":28,"redis_duration_s":0.003849,"redis_read_bytes":36,"redis_write_bytes":10603,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000617,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":680,"redis_queues_calls":2,"redis_queues_duration_s":0.000313,"redis_queues_read_bytes":3,"redis_queues_write_bytes":8315,"redis_queues_metadata_calls":16,"redis_queues_metadata_duration_s":0.002124,"redis_queues_metadata_read_bytes":24,"redis_queues_metadata_write_bytes":1162,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000795,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":446,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035498,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023564,"duration_s":0.044968,"completed_at":"2026-06-29T12:00:21.717Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002409,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1431vm-test-run-nixbot-gitlab> gitlab # [  232.236079] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.718Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::PipelineArtifacts::ExpireArtifactsWorker","jid":"61257563abf19bae7891a341","created_at":"2026-06-29T12:00:14.507Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a644494ee9cf25296a982307a8041dc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:35dc78c13bdf78a98c09fc2795030a4961be87ad882288b77a6a3c9be677e4d5","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.534Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::ExpireArtifactsWorker JID-61257563abf19bae7891a341: start","job_status":"start","queue_duration_s":7.18449,"scheduling_latency_s":7.184508}1432vm-test-run-nixbot-gitlab> gitlab # [  232.249815] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.747Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::PipelineArtifacts::ExpireArtifactsWorker","jid":"61257563abf19bae7891a341","created_at":"2026-06-29T12:00:14.507Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a644494ee9cf25296a982307a8041dc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:35dc78c13bdf78a98c09fc2795030a4961be87ad882288b77a6a3c9be677e4d5","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.534Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::ExpireArtifactsWorker JID-61257563abf19bae7891a341: done: 0.028094 sec","job_status":"done","queue_duration_s":7.18449,"scheduling_latency_s":7.184508,"redis_calls":12,"redis_duration_s":0.001652,"redis_read_bytes":11,"redis_write_bytes":1481,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000287,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000746,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000619,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":548,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00044892,"exclusive_lock_hold_duration_s":0.010568020000000001,"extra.ci_pipeline_artifacts_expire_artifacts_worker.destroyed_pipeline_artifacts_count":0,"duration_s":0.028094,"completed_at":"2026-06-29T12:00:21.746Z","load_balancing_strategy":"primary","db_duration_s":0.005999,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433vm-test-run-nixbot-gitlab> gitlab # [  232.292881] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.748Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"4c40435587fd27a9c6c2d053","created_at":"2026-06-29T12:00:14.552Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.599Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-4c40435587fd27a9c6c2d053: start","job_status":"start","queue_duration_s":7.148861,"scheduling_latency_s":7.14888}1434vm-test-run-nixbot-gitlab> gitlab # [  232.306391] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.842Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"4c40435587fd27a9c6c2d053","created_at":"2026-06-29T12:00:14.552Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.599Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-4c40435587fd27a9c6c2d053: done: 0.094001 sec","job_status":"done","queue_duration_s":7.148861,"scheduling_latency_s":7.14888,"redis_calls":65,"redis_duration_s":0.009349,"redis_read_bytes":115,"redis_write_bytes":41282,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000567,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":656,"redis_queues_calls":2,"redis_queues_duration_s":0.0006,"redis_queues_read_bytes":4,"redis_queues_write_bytes":36985,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.007768,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3374,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000414,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":267,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.083396,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023712,"duration_s":0.094001,"completed_at":"2026-06-29T12:00:21.842Z","load_balancing_strategy":"primary","db_duration_s":0.00062,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435vm-test-run-nixbot-gitlab> gitlab # [  232.350613] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.843Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"b888cedade5299197b5bf7b3","created_at":"2026-06-29T12:00:14.634Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b81c989968dcea7db3f59615d9b083e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.646Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-b888cedade5299197b5bf7b3: start","job_status":"start","queue_duration_s":7.196354,"scheduling_latency_s":7.196374}1436vm-test-run-nixbot-gitlab> gitlab # [  232.364633] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.868Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"b888cedade5299197b5bf7b3","created_at":"2026-06-29T12:00:14.634Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b81c989968dcea7db3f59615d9b083e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:14.646Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-b888cedade5299197b5bf7b3: done: 0.025195 sec","job_status":"done","queue_duration_s":7.196354,"scheduling_latency_s":7.196374,"redis_calls":12,"redis_duration_s":0.001691,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00029,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000733,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000668,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015731,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00051288,"exclusive_lock_hold_duration_s":0.00628074,"duration_s":0.025195,"completed_at":"2026-06-29T12:00:21.868Z","load_balancing_strategy":"primary","db_duration_s":0.00257,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437vm-test-run-nixbot-gitlab> gitlab # [  232.407056] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.869Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"81d5de21eeb4a780d03938ec","created_at":"2026-06-29T12:00:15.887Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-99e32bae84f44424","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"83f33d19602cb566b40cc9845fd4e38f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:15.934Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-81d5de21eeb4a780d03938ec: start","job_status":"start","queue_duration_s":5.935481,"scheduling_latency_s":5.935499}1438vm-test-run-nixbot-gitlab> gitlab # [  232.419796] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.884Z","message":"No existing merge request to be cleaned up."}1439vm-test-run-nixbot-gitlab> gitlab # [  232.421583] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.897Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"81d5de21eeb4a780d03938ec","created_at":"2026-06-29T12:00:15.887Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-99e32bae84f44424","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"83f33d19602cb566b40cc9845fd4e38f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:15.934Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-81d5de21eeb4a780d03938ec: done: 0.027689 sec","job_status":"done","queue_duration_s":5.935481,"scheduling_latency_s":5.935499,"redis_calls":13,"redis_duration_s":0.0019219999999999999,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0003,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000645,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000977,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01825,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026132,"duration_s":0.027689,"completed_at":"2026-06-29T12:00:21.897Z","load_balancing_strategy":"primary","db_duration_s":0.002526,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440vm-test-run-nixbot-gitlab> gitlab # [  232.462251] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.898Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c5c1aa913d951327b76fce63","created_at":"2026-06-29T12:00:15.887Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-99e32bae84f44424","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"83f33d19602cb566b40cc9845fd4e38f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:15.934Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c5c1aa913d951327b76fce63: start","job_status":"start","queue_duration_s":5.96439,"scheduling_latency_s":5.96441}1441vm-test-run-nixbot-gitlab> gitlab # [  232.474839] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.908Z","message":"No existing merge request to be cleaned up."}1442vm-test-run-nixbot-gitlab> gitlab # [  232.476618] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.916Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c5c1aa913d951327b76fce63","created_at":"2026-06-29T12:00:15.887Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-99e32bae84f44424","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"83f33d19602cb566b40cc9845fd4e38f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:15.934Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c5c1aa913d951327b76fce63: done: 0.018171 sec","job_status":"done","queue_duration_s":5.96439,"scheduling_latency_s":5.96441,"redis_calls":10,"redis_duration_s":0.001356,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000599,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000757,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012462,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018171,"completed_at":"2026-06-29T12:00:21.916Z","load_balancing_strategy":"primary","db_duration_s":0.001276,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443vm-test-run-nixbot-gitlab> gitlab # [  232.515250] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.917Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dc174c01afa54a2666a78568","created_at":"2026-06-29T12:00:15.887Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-99e32bae84f44424","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"83f33d19602cb566b40cc9845fd4e38f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:15.934Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dc174c01afa54a2666a78568: start","job_status":"start","queue_duration_s":5.983638,"scheduling_latency_s":5.983656}1444vm-test-run-nixbot-gitlab> gitlab # [  232.527809] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.927Z","message":"No existing merge request to be cleaned up."}1445vm-test-run-nixbot-gitlab> gitlab # [  232.529632] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.935Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dc174c01afa54a2666a78568","created_at":"2026-06-29T12:00:15.887Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-99e32bae84f44424","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"83f33d19602cb566b40cc9845fd4e38f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:15.934Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dc174c01afa54a2666a78568: done: 0.01797 sec","job_status":"done","queue_duration_s":5.983638,"scheduling_latency_s":5.983656,"redis_calls":10,"redis_duration_s":0.0014290000000000001,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000607,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000822,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01797,"completed_at":"2026-06-29T12:00:21.935Z","load_balancing_strategy":"primary","db_duration_s":0.00127,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446vm-test-run-nixbot-gitlab> gitlab # [  232.569129] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.936Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4268c8ba119f1b8252224516","created_at":"2026-06-29T12:00:15.887Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-99e32bae84f44424","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"83f33d19602cb566b40cc9845fd4e38f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:15.934Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4268c8ba119f1b8252224516: start","job_status":"start","queue_duration_s":6.002761,"scheduling_latency_s":6.002779}1447vm-test-run-nixbot-gitlab> gitlab # [  232.581753] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.946Z","message":"No existing merge request to be cleaned up."}1448vm-test-run-nixbot-gitlab> gitlab # [  232.583478] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.954Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4268c8ba119f1b8252224516","created_at":"2026-06-29T12:00:15.887Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-99e32bae84f44424","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"83f33d19602cb566b40cc9845fd4e38f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:15.934Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4268c8ba119f1b8252224516: done: 0.017878 sec","job_status":"done","queue_duration_s":6.002761,"scheduling_latency_s":6.002779,"redis_calls":10,"redis_duration_s":0.001419,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000587,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000832,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012583,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017878,"completed_at":"2026-06-29T12:00:21.954Z","load_balancing_strategy":"primary","db_duration_s":0.001229,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449vm-test-run-nixbot-gitlab> gitlab # [  232.623293] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d6754b3cc9f8b8d7beb278e","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d6754b3cc9f8b8d7beb278e: start","job_status":"start","queue_duration_s":4.962492,"scheduling_latency_s":4.962509}1450vm-test-run-nixbot-gitlab> gitlab # [  232.636083] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d6754b3cc9f8b8d7beb278e","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d6754b3cc9f8b8d7beb278e: done: 0.021421 sec","job_status":"done","queue_duration_s":4.962492,"scheduling_latency_s":4.962509,"redis_calls":16,"redis_duration_s":0.002216,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000295,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000822,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001099,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00048088,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021421,"completed_at":"2026-06-29T12:00:21.977Z","load_balancing_strategy":"primary","db_duration_s":0.000337,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451vm-test-run-nixbot-gitlab> gitlab # [  232.677147] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35bad71a52437e781e5fc3a2","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35bad71a52437e781e5fc3a2: start","job_status":"start","queue_duration_s":4.984994,"scheduling_latency_s":4.985012}1452vm-test-run-nixbot-gitlab> gitlab # [  232.689696] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:21.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35bad71a52437e781e5fc3a2","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35bad71a52437e781e5fc3a2: done: 0.016998 sec","job_status":"done","queue_duration_s":4.984994,"scheduling_latency_s":4.985012,"redis_calls":13,"redis_duration_s":0.0017109999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001166,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016998,"completed_at":"2026-06-29T12:00:21.995Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453vm-test-run-nixbot-gitlab> gitlab # [  232.729137] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50ec78027d234a09d47dd7ed","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50ec78027d234a09d47dd7ed: start","job_status":"start","queue_duration_s":5.014303,"scheduling_latency_s":5.014351}1454vm-test-run-nixbot-gitlab> gitlab # [  232.741384] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.050Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50ec78027d234a09d47dd7ed","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50ec78027d234a09d47dd7ed: done: 0.042475 sec","job_status":"done","queue_duration_s":5.014303,"scheduling_latency_s":5.014351,"redis_calls":13,"redis_duration_s":0.004399,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.042475,"completed_at":"2026-06-29T12:00:22.050Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455vm-test-run-nixbot-gitlab> gitlab # [  232.781593] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb114804942d8f131a782644","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb114804942d8f131a782644: start","job_status":"start","queue_duration_s":5.059727,"scheduling_latency_s":5.059767}1456vm-test-run-nixbot-gitlab> gitlab # [  232.794140] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.082Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb114804942d8f131a782644","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb114804942d8f131a782644: done: 0.02944 sec","job_status":"done","queue_duration_s":5.059727,"scheduling_latency_s":5.059767,"redis_calls":13,"redis_duration_s":0.002883,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00102,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001863,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02944,"completed_at":"2026-06-29T12:00:22.082Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457vm-test-run-nixbot-gitlab> gitlab # [  232.833419] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"480b7b8c1ec4b90d69362d6a","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-480b7b8c1ec4b90d69362d6a: start","job_status":"start","queue_duration_s":5.091059,"scheduling_latency_s":5.091092}1458vm-test-run-nixbot-gitlab> gitlab # [  232.846054] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.105Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"480b7b8c1ec4b90d69362d6a","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-480b7b8c1ec4b90d69362d6a: done: 0.02079 sec","job_status":"done","queue_duration_s":5.091059,"scheduling_latency_s":5.091092,"redis_calls":13,"redis_duration_s":0.002097,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000746,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001351,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013843,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02079,"completed_at":"2026-06-29T12:00:22.105Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459vm-test-run-nixbot-gitlab> gitlab # [  232.885700] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.106Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8a1c5fe409a79254c876350","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8a1c5fe409a79254c876350: start","job_status":"start","queue_duration_s":5.113184,"scheduling_latency_s":5.113205}1460vm-test-run-nixbot-gitlab> gitlab # [  232.898090] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.123Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8a1c5fe409a79254c876350","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8a1c5fe409a79254c876350: done: 0.016997 sec","job_status":"done","queue_duration_s":5.113184,"scheduling_latency_s":5.113205,"redis_calls":13,"redis_duration_s":0.001758,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000655,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011698,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016997,"completed_at":"2026-06-29T12:00:22.123Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461vm-test-run-nixbot-gitlab> gitlab # [  232.937409] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.124Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30b4cb5ae992fd5cf283b895","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30b4cb5ae992fd5cf283b895: start","job_status":"start","queue_duration_s":5.131434,"scheduling_latency_s":5.131453}1462vm-test-run-nixbot-gitlab> gitlab # [  232.949971] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.140Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30b4cb5ae992fd5cf283b895","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30b4cb5ae992fd5cf283b895: done: 0.015702 sec","job_status":"done","queue_duration_s":5.131434,"scheduling_latency_s":5.131453,"redis_calls":13,"redis_duration_s":0.0016,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000598,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015702,"completed_at":"2026-06-29T12:00:22.140Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463vm-test-run-nixbot-gitlab> gitlab # [  232.990186] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad229b00e5ee7f3c9d0ca846","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad229b00e5ee7f3c9d0ca846: start","job_status":"start","queue_duration_s":5.14838,"scheduling_latency_s":5.148398}1464vm-test-run-nixbot-gitlab> gitlab # [  233.002778] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad229b00e5ee7f3c9d0ca846","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad229b00e5ee7f3c9d0ca846: done: 0.014437 sec","job_status":"done","queue_duration_s":5.14838,"scheduling_latency_s":5.148398,"redis_calls":13,"redis_duration_s":0.001471,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000908,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014437,"completed_at":"2026-06-29T12:00:22.156Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465vm-test-run-nixbot-gitlab> gitlab # [  233.042081] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0855448eac603a210d3b28ab","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0855448eac603a210d3b28ab: start","job_status":"start","queue_duration_s":5.164079,"scheduling_latency_s":5.164097}1466vm-test-run-nixbot-gitlab> gitlab # [  233.054760] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0855448eac603a210d3b28ab","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0855448eac603a210d3b28ab: done: 0.015044 sec","job_status":"done","queue_duration_s":5.164079,"scheduling_latency_s":5.164097,"redis_calls":13,"redis_duration_s":0.0014780000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015044,"completed_at":"2026-06-29T12:00:22.172Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1467vm-test-run-nixbot-gitlab> gitlab # [  233.094156] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.173Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cde7e21fb3929cbd30ed710","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cde7e21fb3929cbd30ed710: start","job_status":"start","queue_duration_s":5.180194,"scheduling_latency_s":5.180211}1468vm-test-run-nixbot-gitlab> gitlab # [  233.106470] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.188Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cde7e21fb3929cbd30ed710","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cde7e21fb3929cbd30ed710: done: 0.014991 sec","job_status":"done","queue_duration_s":5.180194,"scheduling_latency_s":5.180211,"redis_calls":13,"redis_duration_s":0.001505,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000972,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014991,"completed_at":"2026-06-29T12:00:22.188Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1469vm-test-run-nixbot-gitlab> gitlab # [  233.145689] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.189Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c35de0c642f732b58860dd71","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c35de0c642f732b58860dd71: start","job_status":"start","queue_duration_s":5.196208,"scheduling_latency_s":5.196225}1470vm-test-run-nixbot-gitlab> gitlab # [  233.158284] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.204Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c35de0c642f732b58860dd71","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c35de0c642f732b58860dd71: done: 0.014722 sec","job_status":"done","queue_duration_s":5.196208,"scheduling_latency_s":5.196225,"redis_calls":13,"redis_duration_s":0.001636,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014722,"completed_at":"2026-06-29T12:00:22.204Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471vm-test-run-nixbot-gitlab> gitlab # [  233.198523] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e0e5a7922a8f5b55ba5c8e4","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e0e5a7922a8f5b55ba5c8e4: start","job_status":"start","queue_duration_s":5.211977,"scheduling_latency_s":5.211994}1472vm-test-run-nixbot-gitlab> gitlab # [  233.211738] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e0e5a7922a8f5b55ba5c8e4","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e0e5a7922a8f5b55ba5c8e4: done: 0.01493 sec","job_status":"done","queue_duration_s":5.211977,"scheduling_latency_s":5.211994,"redis_calls":13,"redis_duration_s":0.0014789999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01493,"completed_at":"2026-06-29T12:00:22.220Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473vm-test-run-nixbot-gitlab> gitlab # [  233.252229] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.221Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa982b81dd1a71b074180267","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa982b81dd1a71b074180267: start","job_status":"start","queue_duration_s":5.228045,"scheduling_latency_s":5.228063}1474vm-test-run-nixbot-gitlab> gitlab # [  233.264669] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.237Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa982b81dd1a71b074180267","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa982b81dd1a71b074180267: done: 0.015922 sec","job_status":"done","queue_duration_s":5.228045,"scheduling_latency_s":5.228063,"redis_calls":13,"redis_duration_s":0.001571,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015922,"completed_at":"2026-06-29T12:00:22.237Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1475vm-test-run-nixbot-gitlab> gitlab # [  233.304429] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.238Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be53dd0a705cd77fff851db0","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be53dd0a705cd77fff851db0: start","job_status":"start","queue_duration_s":5.245381,"scheduling_latency_s":5.245399}1476vm-test-run-nixbot-gitlab> gitlab # [  233.316761] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.270Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be53dd0a705cd77fff851db0","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be53dd0a705cd77fff851db0: done: 0.032097 sec","job_status":"done","queue_duration_s":5.245381,"scheduling_latency_s":5.245399,"redis_calls":13,"redis_duration_s":0.015314,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.014303,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032097,"completed_at":"2026-06-29T12:00:22.270Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477vm-test-run-nixbot-gitlab> gitlab # [  233.356832] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d51be5a90090841bb081bd2","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d51be5a90090841bb081bd2: start","job_status":"start","queue_duration_s":5.2789,"scheduling_latency_s":5.27892}1478vm-test-run-nixbot-gitlab> gitlab # [  233.369944] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d51be5a90090841bb081bd2","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d51be5a90090841bb081bd2: done: 0.028032 sec","job_status":"done","queue_duration_s":5.2789,"scheduling_latency_s":5.27892,"redis_calls":13,"redis_duration_s":0.0036249999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001123,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002502,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028032,"completed_at":"2026-06-29T12:00:22.300Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479vm-test-run-nixbot-gitlab> gitlab # [  233.409862] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5dd215918baf86ff3e0f990","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5dd215918baf86ff3e0f990: start","job_status":"start","queue_duration_s":5.310118,"scheduling_latency_s":5.310163}1480vm-test-run-nixbot-gitlab> gitlab # [  233.422941] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5dd215918baf86ff3e0f990","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5dd215918baf86ff3e0f990: done: 0.035467 sec","job_status":"done","queue_duration_s":5.310118,"scheduling_latency_s":5.310163,"redis_calls":13,"redis_duration_s":0.004364,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001297,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035467,"completed_at":"2026-06-29T12:00:22.338Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481vm-test-run-nixbot-gitlab> gitlab # [  233.463149] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.341Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68d58dc5ca7f75eecfcb1ad1","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68d58dc5ca7f75eecfcb1ad1: start","job_status":"start","queue_duration_s":5.347916,"scheduling_latency_s":5.347949}1482vm-test-run-nixbot-gitlab> gitlab # [  233.476387] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.368Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68d58dc5ca7f75eecfcb1ad1","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68d58dc5ca7f75eecfcb1ad1: done: 0.026934 sec","job_status":"done","queue_duration_s":5.347916,"scheduling_latency_s":5.347949,"redis_calls":13,"redis_duration_s":0.003963,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001217,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002746,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016655,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026934,"completed_at":"2026-06-29T12:00:22.368Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1483vm-test-run-nixbot-gitlab> gitlab # [  233.515829] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.370Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31db74bb3cfb90885b4c21f6","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31db74bb3cfb90885b4c21f6: start","job_status":"start","queue_duration_s":5.376842,"scheduling_latency_s":5.376871}1484vm-test-run-nixbot-gitlab> gitlab # [  233.528341] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.393Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31db74bb3cfb90885b4c21f6","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31db74bb3cfb90885b4c21f6: done: 0.02345 sec","job_status":"done","queue_duration_s":5.376842,"scheduling_latency_s":5.376871,"redis_calls":13,"redis_duration_s":0.0032640000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001087,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002177,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02345,"completed_at":"2026-06-29T12:00:22.393Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485vm-test-run-nixbot-gitlab> gitlab # [  233.567977] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ace49293590e28e04dc3723e","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ace49293590e28e04dc3723e: start","job_status":"start","queue_duration_s":5.401927,"scheduling_latency_s":5.401949}1486vm-test-run-nixbot-gitlab> gitlab # [  233.581343] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.414Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ace49293590e28e04dc3723e","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ace49293590e28e04dc3723e: done: 0.019281 sec","job_status":"done","queue_duration_s":5.401927,"scheduling_latency_s":5.401949,"redis_calls":13,"redis_duration_s":0.002968,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000997,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019281,"completed_at":"2026-06-29T12:00:22.414Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487vm-test-run-nixbot-gitlab> gitlab # [  233.620625] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b592187ebd6a5b8acc1e2d02","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b592187ebd6a5b8acc1e2d02: start","job_status":"start","queue_duration_s":5.423872,"scheduling_latency_s":5.423915}1488vm-test-run-nixbot-gitlab> gitlab # [  233.633260] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.455Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b592187ebd6a5b8acc1e2d02","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b592187ebd6a5b8acc1e2d02: done: 0.038437 sec","job_status":"done","queue_duration_s":5.423872,"scheduling_latency_s":5.423915,"redis_calls":13,"redis_duration_s":0.0039570000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001458,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002499,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02649,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038437,"completed_at":"2026-06-29T12:00:22.455Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489vm-test-run-nixbot-gitlab> gitlab # [  233.672967] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.458Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa36efda204115f4fde98cd0","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa36efda204115f4fde98cd0: start","job_status":"start","queue_duration_s":5.464872,"scheduling_latency_s":5.464918}1490vm-test-run-nixbot-gitlab> gitlab # [  233.685386] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.776Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa36efda204115f4fde98cd0","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa36efda204115f4fde98cd0: done: 0.318623 sec","job_status":"done","queue_duration_s":5.464872,"scheduling_latency_s":5.464918,"redis_calls":13,"redis_duration_s":0.003737,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001407,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00233,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025639,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.318623,"completed_at":"2026-06-29T12:00:22.776Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491vm-test-run-nixbot-gitlab> gitlab # [  233.724678] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.780Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63bba7b0534e4aa7881c4140","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63bba7b0534e4aa7881c4140: start","job_status":"start","queue_duration_s":5.786965,"scheduling_latency_s":5.787061}1492vm-test-run-nixbot-gitlab> gitlab # [  233.737067] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.818Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63bba7b0534e4aa7881c4140","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63bba7b0534e4aa7881c4140: done: 0.038295 sec","job_status":"done","queue_duration_s":5.786965,"scheduling_latency_s":5.787061,"redis_calls":13,"redis_duration_s":0.005116,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001302,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003814,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038295,"completed_at":"2026-06-29T12:00:22.818Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493vm-test-run-nixbot-gitlab> gitlab # [  233.777243] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.821Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ffcaaa41b09d3a3781db015","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ffcaaa41b09d3a3781db015: start","job_status":"start","queue_duration_s":5.828004,"scheduling_latency_s":5.828046}1494vm-test-run-nixbot-gitlab> gitlab # [  233.789891] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ffcaaa41b09d3a3781db015","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ffcaaa41b09d3a3781db015: done: 0.02828 sec","job_status":"done","queue_duration_s":5.828004,"scheduling_latency_s":5.828046,"redis_calls":13,"redis_duration_s":0.0034939999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001155,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002339,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01849,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02828,"completed_at":"2026-06-29T12:00:22.849Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495vm-test-run-nixbot-gitlab> gitlab # [  233.829310] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.851Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb736ff60f2ccdb9644d759d","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb736ff60f2ccdb9644d759d: start","job_status":"start","queue_duration_s":5.85818,"scheduling_latency_s":5.858213}1496vm-test-run-nixbot-gitlab> gitlab # [  233.841997] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb736ff60f2ccdb9644d759d","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb736ff60f2ccdb9644d759d: done: 0.022315 sec","job_status":"done","queue_duration_s":5.85818,"scheduling_latency_s":5.858213,"redis_calls":13,"redis_duration_s":0.003699,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002646,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022315,"completed_at":"2026-06-29T12:00:22.873Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497vm-test-run-nixbot-gitlab> gitlab # [  233.881285] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.875Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36a7904cd8e27c533628d521","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36a7904cd8e27c533628d521: start","job_status":"start","queue_duration_s":5.882127,"scheduling_latency_s":5.882148}1498vm-test-run-nixbot-gitlab> gitlab # [  233.893858] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.897Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36a7904cd8e27c533628d521","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36a7904cd8e27c533628d521: done: 0.021658 sec","job_status":"done","queue_duration_s":5.882127,"scheduling_latency_s":5.882148,"redis_calls":13,"redis_duration_s":0.0031130000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001077,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021658,"completed_at":"2026-06-29T12:00:22.897Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1499vm-test-run-nixbot-gitlab> gitlab # [  233.933302] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"608fabb48fd6e8cc160baa6a","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-608fabb48fd6e8cc160baa6a: start","job_status":"start","queue_duration_s":5.905492,"scheduling_latency_s":5.905514}1500vm-test-run-nixbot-gitlab> gitlab # [  233.945869] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"608fabb48fd6e8cc160baa6a","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-608fabb48fd6e8cc160baa6a: done: 0.019132 sec","job_status":"done","queue_duration_s":5.905492,"scheduling_latency_s":5.905514,"redis_calls":13,"redis_duration_s":0.002876,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000978,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001898,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019132,"completed_at":"2026-06-29T12:00:22.917Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1501vm-test-run-nixbot-gitlab> gitlab # [  233.986137] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:22.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e5ee38d8791ed433b10056f","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e5ee38d8791ed433b10056f: start","job_status":"start","queue_duration_s":5.926122,"scheduling_latency_s":5.926141}1502vm-test-run-nixbot-gitlab> gitlab # [  233.998640] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e5ee38d8791ed433b10056f","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e5ee38d8791ed433b10056f: done: 0.15038 sec","job_status":"done","queue_duration_s":5.926122,"scheduling_latency_s":5.926141,"redis_calls":13,"redis_duration_s":0.003185,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001383,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001802,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.15038,"completed_at":"2026-06-29T12:00:23.069Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1503vm-test-run-nixbot-gitlab> gitlab # [  234.038152] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"963dcdb881717a593d8c9b6b","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-963dcdb881717a593d8c9b6b: start","job_status":"start","queue_duration_s":6.080311,"scheduling_latency_s":6.080353}1504vm-test-run-nixbot-gitlab> gitlab # [  234.050483] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"963dcdb881717a593d8c9b6b","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-963dcdb881717a593d8c9b6b: done: 0.041085 sec","job_status":"done","queue_duration_s":6.080311,"scheduling_latency_s":6.080353,"redis_calls":13,"redis_duration_s":0.005293,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001987,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003306,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.041085,"completed_at":"2026-06-29T12:00:23.114Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505vm-test-run-nixbot-gitlab> gitlab # [  234.089898] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.119Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d6e2027b6d46d53f0852e99","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d6e2027b6d46d53f0852e99: start","job_status":"start","queue_duration_s":6.126411,"scheduling_latency_s":6.126456}1506vm-test-run-nixbot-gitlab> gitlab # [  234.102591] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d6e2027b6d46d53f0852e99","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d6e2027b6d46d53f0852e99: done: 0.037516 sec","job_status":"done","queue_duration_s":6.126411,"scheduling_latency_s":6.126456,"redis_calls":13,"redis_duration_s":0.004353,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001473,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00288,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.037516,"completed_at":"2026-06-29T12:00:23.157Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507vm-test-run-nixbot-gitlab> gitlab # [  234.142075] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.159Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"578e8b2d44acb85efe934568","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-578e8b2d44acb85efe934568: start","job_status":"start","queue_duration_s":6.166255,"scheduling_latency_s":6.166286}1508vm-test-run-nixbot-gitlab> gitlab # [  234.154724] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"578e8b2d44acb85efe934568","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-578e8b2d44acb85efe934568: done: 0.030443 sec","job_status":"done","queue_duration_s":6.166255,"scheduling_latency_s":6.166286,"redis_calls":13,"redis_duration_s":0.004261,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001139,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003122,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017148,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030443,"completed_at":"2026-06-29T12:00:23.189Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509vm-test-run-nixbot-gitlab> gitlab # [  234.195053] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31e585032a58a8ac6d4a8588","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31e585032a58a8ac6d4a8588: start","job_status":"start","queue_duration_s":6.198739,"scheduling_latency_s":6.198765}1510vm-test-run-nixbot-gitlab> gitlab # [  234.207798] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.217Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31e585032a58a8ac6d4a8588","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31e585032a58a8ac6d4a8588: done: 0.024958 sec","job_status":"done","queue_duration_s":6.198739,"scheduling_latency_s":6.198765,"redis_calls":13,"redis_duration_s":0.003506,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00117,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002336,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016752,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024958,"completed_at":"2026-06-29T12:00:23.216Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1511vm-test-run-nixbot-gitlab> gitlab # [  234.247872] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a23362ff88784e93ce4a470b","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a23362ff88784e93ce4a470b: start","job_status":"start","queue_duration_s":6.226706,"scheduling_latency_s":6.226732}1512vm-test-run-nixbot-gitlab> gitlab # [  234.260262] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.241Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a23362ff88784e93ce4a470b","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a23362ff88784e93ce4a470b: done: 0.021683 sec","job_status":"done","queue_duration_s":6.226706,"scheduling_latency_s":6.226732,"redis_calls":13,"redis_duration_s":0.0038850000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001868,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021683,"completed_at":"2026-06-29T12:00:23.241Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1513vm-test-run-nixbot-gitlab> gitlab # [  234.300131] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.243Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93403490f975541bed22e9e3","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93403490f975541bed22e9e3: start","job_status":"start","queue_duration_s":6.249974,"scheduling_latency_s":6.249994}1514vm-test-run-nixbot-gitlab> gitlab # [  234.312283] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.262Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93403490f975541bed22e9e3","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93403490f975541bed22e9e3: done: 0.018735 sec","job_status":"done","queue_duration_s":6.249974,"scheduling_latency_s":6.249994,"redis_calls":13,"redis_duration_s":0.003804,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000997,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002807,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018735,"completed_at":"2026-06-29T12:00:23.262Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1515vm-test-run-nixbot-gitlab> gitlab # [  234.352765] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.263Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"386eca500b07bdf92b50fcf6","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-386eca500b07bdf92b50fcf6: start","job_status":"start","queue_duration_s":6.270204,"scheduling_latency_s":6.270222}1516vm-test-run-nixbot-gitlab> gitlab # [  234.365463] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"386eca500b07bdf92b50fcf6","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-386eca500b07bdf92b50fcf6: done: 0.021433 sec","job_status":"done","queue_duration_s":6.270204,"scheduling_latency_s":6.270222,"redis_calls":13,"redis_duration_s":0.007048,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000795,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006253,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011396,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021433,"completed_at":"2026-06-29T12:00:23.284Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1517vm-test-run-nixbot-gitlab> gitlab # [  234.405889] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca3e6d1c0b6e47b305325542","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca3e6d1c0b6e47b305325542: start","job_status":"start","queue_duration_s":6.292904,"scheduling_latency_s":6.292923}1518vm-test-run-nixbot-gitlab> gitlab # [  234.418331] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca3e6d1c0b6e47b305325542","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca3e6d1c0b6e47b305325542: done: 0.016433 sec","job_status":"done","queue_duration_s":6.292904,"scheduling_latency_s":6.292923,"redis_calls":13,"redis_duration_s":0.0016649999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000641,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016433,"completed_at":"2026-06-29T12:00:23.302Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1519vm-test-run-nixbot-gitlab> gitlab # [  234.458103] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6642db2604402f66eb029060","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6642db2604402f66eb029060: start","job_status":"start","queue_duration_s":6.311604,"scheduling_latency_s":6.311622}1520vm-test-run-nixbot-gitlab> gitlab # [  234.471076] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6642db2604402f66eb029060","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6642db2604402f66eb029060: done: 0.015599 sec","job_status":"done","queue_duration_s":6.311604,"scheduling_latency_s":6.311622,"redis_calls":13,"redis_duration_s":0.00152,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015599,"completed_at":"2026-06-29T12:00:23.320Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1521vm-test-run-nixbot-gitlab> gitlab # [  234.510570] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33072e86b1b42b0ad5929607","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33072e86b1b42b0ad5929607: start","job_status":"start","queue_duration_s":6.328301,"scheduling_latency_s":6.328318}1522vm-test-run-nixbot-gitlab> gitlab # [  234.523338] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.337Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33072e86b1b42b0ad5929607","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33072e86b1b42b0ad5929607: done: 0.015859 sec","job_status":"done","queue_duration_s":6.328301,"scheduling_latency_s":6.328318,"redis_calls":13,"redis_duration_s":0.002134,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000571,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001563,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015859,"completed_at":"2026-06-29T12:00:23.337Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1523vm-test-run-nixbot-gitlab> gitlab # [  234.563599] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09df2b287f5de4d961c8be95","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09df2b287f5de4d961c8be95: start","job_status":"start","queue_duration_s":6.34522,"scheduling_latency_s":6.345237}1524vm-test-run-nixbot-gitlab> gitlab # [  234.576071] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09df2b287f5de4d961c8be95","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09df2b287f5de4d961c8be95: done: 0.014699 sec","job_status":"done","queue_duration_s":6.34522,"scheduling_latency_s":6.345237,"redis_calls":13,"redis_duration_s":0.0014520000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014699,"completed_at":"2026-06-29T12:00:23.353Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1525vm-test-run-nixbot-gitlab> gitlab # [  234.616807] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fefab01d37d39cd44e1695d3","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fefab01d37d39cd44e1695d3: start","job_status":"start","queue_duration_s":6.361865,"scheduling_latency_s":6.361884}1526vm-test-run-nixbot-gitlab> gitlab # [  234.629495] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.370Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fefab01d37d39cd44e1695d3","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fefab01d37d39cd44e1695d3: done: 0.015453 sec","job_status":"done","queue_duration_s":6.361865,"scheduling_latency_s":6.361884,"redis_calls":13,"redis_duration_s":0.0014919999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000967,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010492,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015453,"completed_at":"2026-06-29T12:00:23.370Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1527vm-test-run-nixbot-gitlab> gitlab # [  234.669092] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.371Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"573ce810977ed8547654c071","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-573ce810977ed8547654c071: start","job_status":"start","queue_duration_s":6.378405,"scheduling_latency_s":6.378422}1528vm-test-run-nixbot-gitlab> gitlab # [  234.681684] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"573ce810977ed8547654c071","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-573ce810977ed8547654c071: done: 0.014805 sec","job_status":"done","queue_duration_s":6.378405,"scheduling_latency_s":6.378422,"redis_calls":13,"redis_duration_s":0.001533,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014805,"completed_at":"2026-06-29T12:00:23.386Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1529vm-test-run-nixbot-gitlab> gitlab # [  234.721174] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4302274f73e5cc1f71ad376e","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4302274f73e5cc1f71ad376e: start","job_status":"start","queue_duration_s":6.394265,"scheduling_latency_s":6.394282}1530vm-test-run-nixbot-gitlab> gitlab # [  234.733465] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.402Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4302274f73e5cc1f71ad376e","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4302274f73e5cc1f71ad376e: done: 0.015019 sec","job_status":"done","queue_duration_s":6.394265,"scheduling_latency_s":6.394282,"redis_calls":13,"redis_duration_s":0.0015240000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015019,"completed_at":"2026-06-29T12:00:23.402Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1531vm-test-run-nixbot-gitlab> gitlab # [  234.773097] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.403Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79998a718700bf1e1e6e0683","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79998a718700bf1e1e6e0683: start","job_status":"start","queue_duration_s":6.410396,"scheduling_latency_s":6.410413}1532vm-test-run-nixbot-gitlab> gitlab # [  234.785643] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79998a718700bf1e1e6e0683","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79998a718700bf1e1e6e0683: done: 0.014952 sec","job_status":"done","queue_duration_s":6.410396,"scheduling_latency_s":6.410413,"redis_calls":13,"redis_duration_s":0.00147,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009481,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014952,"completed_at":"2026-06-29T12:00:23.418Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1533vm-test-run-nixbot-gitlab> gitlab # [  234.826103] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.420Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a2d3a5f84b66456c8b6762d","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a2d3a5f84b66456c8b6762d: start","job_status":"start","queue_duration_s":6.427313,"scheduling_latency_s":6.42733}1534vm-test-run-nixbot-gitlab> gitlab # [  234.838805] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.435Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a2d3a5f84b66456c8b6762d","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a2d3a5f84b66456c8b6762d: done: 0.014803 sec","job_status":"done","queue_duration_s":6.427313,"scheduling_latency_s":6.42733,"redis_calls":13,"redis_duration_s":0.001441,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014803,"completed_at":"2026-06-29T12:00:23.435Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1535vm-test-run-nixbot-gitlab> gitlab # [  234.878437] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.436Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6c1d38606f13d66baafe86c","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6c1d38606f13d66baafe86c: start","job_status":"start","queue_duration_s":6.443318,"scheduling_latency_s":6.443335}1536vm-test-run-nixbot-gitlab> gitlab # [  234.891156] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.451Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6c1d38606f13d66baafe86c","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6c1d38606f13d66baafe86c: done: 0.015021 sec","job_status":"done","queue_duration_s":6.443318,"scheduling_latency_s":6.443335,"redis_calls":13,"redis_duration_s":0.0014919999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000962,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015021,"completed_at":"2026-06-29T12:00:23.451Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1537vm-test-run-nixbot-gitlab> gitlab # [  234.931126] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e8828185ff8239168e836091","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e8828185ff8239168e836091: start","job_status":"start","queue_duration_s":6.459425,"scheduling_latency_s":6.459443}1538vm-test-run-nixbot-gitlab> gitlab # [  234.943879] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e8828185ff8239168e836091","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e8828185ff8239168e836091: done: 0.014628 sec","job_status":"done","queue_duration_s":6.459425,"scheduling_latency_s":6.459443,"redis_calls":13,"redis_duration_s":0.001425,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014628,"completed_at":"2026-06-29T12:00:23.467Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1539vm-test-run-nixbot-gitlab> gitlab # [  234.983359] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7bb020e9ccf9ac411cc81ef","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7bb020e9ccf9ac411cc81ef: start","job_status":"start","queue_duration_s":6.476136,"scheduling_latency_s":6.476155}1540vm-test-run-nixbot-gitlab> gitlab # [  234.996116] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7bb020e9ccf9ac411cc81ef","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7bb020e9ccf9ac411cc81ef: done: 0.015269 sec","job_status":"done","queue_duration_s":6.476136,"scheduling_latency_s":6.476155,"redis_calls":13,"redis_duration_s":0.001461,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000916,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015269,"completed_at":"2026-06-29T12:00:23.484Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1541vm-test-run-nixbot-gitlab> gitlab # [  235.036606] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d0ebb56e55ab73fc0d9e6ef","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d0ebb56e55ab73fc0d9e6ef: start","job_status":"start","queue_duration_s":6.492502,"scheduling_latency_s":6.492519}1542vm-test-run-nixbot-gitlab> gitlab # [  235.048922] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.500Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d0ebb56e55ab73fc0d9e6ef","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d0ebb56e55ab73fc0d9e6ef: done: 0.015014 sec","job_status":"done","queue_duration_s":6.492502,"scheduling_latency_s":6.492519,"redis_calls":13,"redis_duration_s":0.002242,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001708,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010559,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015014,"completed_at":"2026-06-29T12:00:23.500Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1543vm-test-run-nixbot-gitlab> gitlab # [  235.088590] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.501Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0da2eabf840025b4406f1f08","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0da2eabf840025b4406f1f08: start","job_status":"start","queue_duration_s":6.508568,"scheduling_latency_s":6.508586}1544vm-test-run-nixbot-gitlab> gitlab # [  235.101165] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.516Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0da2eabf840025b4406f1f08","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0da2eabf840025b4406f1f08: done: 0.014728 sec","job_status":"done","queue_duration_s":6.508568,"scheduling_latency_s":6.508586,"redis_calls":13,"redis_duration_s":0.001424,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000905,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014728,"completed_at":"2026-06-29T12:00:23.516Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1545vm-test-run-nixbot-gitlab> gitlab # [  235.141144] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.517Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b88d1d731e52d0837dc91590","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b88d1d731e52d0837dc91590: start","job_status":"start","queue_duration_s":6.524414,"scheduling_latency_s":6.524432}1546vm-test-run-nixbot-gitlab> gitlab # [  235.153825] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.533Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b88d1d731e52d0837dc91590","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b88d1d731e52d0837dc91590: done: 0.015754 sec","job_status":"done","queue_duration_s":6.524414,"scheduling_latency_s":6.524432,"redis_calls":13,"redis_duration_s":0.00146,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000912,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015754,"completed_at":"2026-06-29T12:00:23.533Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1547vm-test-run-nixbot-gitlab> gitlab # [  235.193288] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.534Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d69dbf923e337212ac32d90c","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d69dbf923e337212ac32d90c: start","job_status":"start","queue_duration_s":6.541314,"scheduling_latency_s":6.541333}1548vm-test-run-nixbot-gitlab> gitlab # [  235.206503] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.549Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d69dbf923e337212ac32d90c","created_at":"2026-06-29T12:00:16.463Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-8a5107d964ed47c7","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1b063b1420617c1ae8139e2d366fb0a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:16.993Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d69dbf923e337212ac32d90c: done: 0.015152 sec","job_status":"done","queue_duration_s":6.541314,"scheduling_latency_s":6.541333,"redis_calls":13,"redis_duration_s":0.001483,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000585,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000898,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010677,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015152,"completed_at":"2026-06-29T12:00:23.549Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1549vm-test-run-nixbot-gitlab> gitlab # [  235.246453] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.550Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"f087d94b647650ecfe61d2ba","created_at":"2026-06-29T12:00:20.864Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-7cc027b36ec34cbf","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"cc43998a27f4c0e3ed2e5b862183dbfc","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:20.885Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-f087d94b647650ecfe61d2ba: start","job_status":"start","queue_duration_s":2.665137,"scheduling_latency_s":2.665155}1550vm-test-run-nixbot-gitlab> gitlab # [  235.261737] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.580Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"f087d94b647650ecfe61d2ba","created_at":"2026-06-29T12:00:20.864Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-7cc027b36ec34cbf","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"cc43998a27f4c0e3ed2e5b862183dbfc","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:20.885Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-f087d94b647650ecfe61d2ba: done: 0.029145 sec","job_status":"done","queue_duration_s":2.665137,"scheduling_latency_s":2.665155,"redis_calls":10,"redis_duration_s":0.0016970000000000002,"redis_read_bytes":9,"redis_write_bytes":1117,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000321,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001085,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000291,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":222,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018389,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025726,"duration_s":0.029145,"completed_at":"2026-06-29T12:00:23.580Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003101,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1551vm-test-run-nixbot-gitlab> gitlab # [  235.305084] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.581Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"e02e16bf7fc4d1034c88ce79","created_at":"2026-06-29T12:00:20.887Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-7cc027b36ec34cbf","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"cc43998a27f4c0e3ed2e5b862183dbfc","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:20.895Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-e02e16bf7fc4d1034c88ce79: start","job_status":"start","queue_duration_s":2.685892,"scheduling_latency_s":2.685909}1552vm-test-run-nixbot-gitlab> gitlab # [  235.320290] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.601Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"e02e16bf7fc4d1034c88ce79","created_at":"2026-06-29T12:00:20.887Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-7cc027b36ec34cbf","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"cc43998a27f4c0e3ed2e5b862183dbfc","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-06-29T12:00:20.895Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-e02e16bf7fc4d1034c88ce79: done: 0.020448 sec","job_status":"done","queue_duration_s":2.685892,"scheduling_latency_s":2.685909,"redis_calls":10,"redis_duration_s":0.001356,"redis_read_bytes":9,"redis_write_bytes":1133,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000279,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000785,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000292,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024588,"duration_s":0.020448,"completed_at":"2026-06-29T12:00:23.601Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000696,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1553vm-test-run-nixbot-gitlab> gitlab # [  235.363206] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.602Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"c6296de5957b1ed7579786fc","created_at":"2026-06-29T12:00:21.620Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-1eeb1113387542ca","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"ec92a78e13dcc629aef582875d7dbfe8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.633Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-c6296de5957b1ed7579786fc: start","job_status":"start","queue_duration_s":1.969162,"scheduling_latency_s":1.96918}1554vm-test-run-nixbot-gitlab> gitlab # [  235.376101] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.633Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"c6296de5957b1ed7579786fc","created_at":"2026-06-29T12:00:21.620Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-1eeb1113387542ca","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"ec92a78e13dcc629aef582875d7dbfe8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.633Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-c6296de5957b1ed7579786fc: done: 0.030598 sec","job_status":"done","queue_duration_s":1.969162,"scheduling_latency_s":1.96918,"redis_calls":18,"redis_duration_s":0.002505,"redis_read_bytes":39,"redis_write_bytes":2116,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000279,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000623,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":471,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.001603,"redis_shared_state_read_bytes":34,"redis_shared_state_write_bytes":1324,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024314,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":2,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":2,"duration_s":0.030598,"completed_at":"2026-06-29T12:00:23.633Z","load_balancing_strategy":"primary","db_duration_s":0.00273,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1555vm-test-run-nixbot-gitlab> gitlab # [  235.420869] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"3c97395acb42cf35e81f8b8f","created_at":"2026-06-29T12:00:21.620Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-1eeb1113387542ca","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"ec92a78e13dcc629aef582875d7dbfe8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.633Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-3c97395acb42cf35e81f8b8f: start","job_status":"start","queue_duration_s":2.000832,"scheduling_latency_s":2.00085}1556vm-test-run-nixbot-gitlab> gitlab # [  235.433310] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.655Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"3c97395acb42cf35e81f8b8f","created_at":"2026-06-29T12:00:21.620Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-1eeb1113387542ca","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"ec92a78e13dcc629aef582875d7dbfe8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.633Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-3c97395acb42cf35e81f8b8f: done: 0.021382 sec","job_status":"done","queue_duration_s":2.000832,"scheduling_latency_s":2.00085,"redis_calls":15,"redis_duration_s":0.001814,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.001244,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":0,"duration_s":0.021382,"completed_at":"2026-06-29T12:00:23.655Z","load_balancing_strategy":"primary","db_duration_s":0.00208,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1557vm-test-run-nixbot-gitlab> gitlab # [  235.475496] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.657Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"7717fcbe0a5ba4b68b414e2a","created_at":"2026-06-29T12:00:21.620Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-1eeb1113387542ca","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"ec92a78e13dcc629aef582875d7dbfe8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.633Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-7717fcbe0a5ba4b68b414e2a: start","job_status":"start","queue_duration_s":2.023361,"scheduling_latency_s":2.02338}1558vm-test-run-nixbot-gitlab> gitlab # [  235.488106] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"7717fcbe0a5ba4b68b414e2a","created_at":"2026-06-29T12:00:21.620Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-1eeb1113387542ca","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"ec92a78e13dcc629aef582875d7dbfe8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.633Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-7717fcbe0a5ba4b68b414e2a: done: 0.021152 sec","job_status":"done","queue_duration_s":2.023361,"scheduling_latency_s":2.02338,"redis_calls":15,"redis_duration_s":0.00214,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.001265,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":1,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":1,"duration_s":0.021152,"completed_at":"2026-06-29T12:00:23.678Z","load_balancing_strategy":"primary","db_duration_s":0.002106,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1559vm-test-run-nixbot-gitlab> gitlab # [  235.530697] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"e31a86416f110c3da00a189e","created_at":"2026-06-29T12:00:21.620Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-1eeb1113387542ca","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"ec92a78e13dcc629aef582875d7dbfe8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.633Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-e31a86416f110c3da00a189e: start","job_status":"start","queue_duration_s":2.045954,"scheduling_latency_s":2.046842}1560vm-test-run-nixbot-gitlab> gitlab # [  235.543475] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.702Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"e31a86416f110c3da00a189e","created_at":"2026-06-29T12:00:21.620Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-1eeb1113387542ca","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"ec92a78e13dcc629aef582875d7dbfe8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.633Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-e31a86416f110c3da00a189e: done: 0.023198 sec","job_status":"done","queue_duration_s":2.045954,"scheduling_latency_s":2.046842,"redis_calls":15,"redis_duration_s":0.003215,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001022,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.002193,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":0,"duration_s":0.023198,"completed_at":"2026-06-29T12:00:23.702Z","load_balancing_strategy":"primary","db_duration_s":0.002023,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1561vm-test-run-nixbot-gitlab> gitlab # [  235.586359] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"665461bb3ea33096d988fb33","created_at":"2026-06-29T12:00:21.620Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-1eeb1113387542ca","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"ec92a78e13dcc629aef582875d7dbfe8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.633Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-665461bb3ea33096d988fb33: start","job_status":"start","queue_duration_s":2.07052,"scheduling_latency_s":2.070539}1562vm-test-run-nixbot-gitlab> gitlab # [  235.598964] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.725Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"665461bb3ea33096d988fb33","created_at":"2026-06-29T12:00:21.620Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-1eeb1113387542ca","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"ec92a78e13dcc629aef582875d7dbfe8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.633Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-665461bb3ea33096d988fb33: done: 0.02165 sec","job_status":"done","queue_duration_s":2.07052,"scheduling_latency_s":2.070539,"redis_calls":15,"redis_duration_s":0.002917,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.002016,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014456,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":3,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":3,"duration_s":0.02165,"completed_at":"2026-06-29T12:00:23.725Z","load_balancing_strategy":"primary","db_duration_s":0.002096,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1563vm-test-run-nixbot-gitlab> gitlab # [  235.642327] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"4222026780fb63abbab16da3","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-4222026780fb63abbab16da3: start","job_status":"start","queue_duration_s":2.016805,"scheduling_latency_s":2.016823}1564vm-test-run-nixbot-gitlab> gitlab # [  235.655894] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.770Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"4222026780fb63abbab16da3","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-4222026780fb63abbab16da3: done: 0.043031 sec","job_status":"done","queue_duration_s":2.016805,"scheduling_latency_s":2.016823,"redis_calls":16,"redis_duration_s":0.004126,"redis_read_bytes":14,"redis_write_bytes":1526,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000501,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001139,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":423,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002486,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":802,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000326,"duration_s":0.043031,"completed_at":"2026-06-29T12:00:23.770Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008645,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1565vm-test-run-nixbot-gitlab> gitlab # [  235.697581] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.771Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"76f0f9ad99a0c9c1be936f54","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-76f0f9ad99a0c9c1be936f54: start","job_status":"start","queue_duration_s":2.061248,"scheduling_latency_s":2.061265}1566vm-test-run-nixbot-gitlab> gitlab # [  235.710814] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"76f0f9ad99a0c9c1be936f54","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-76f0f9ad99a0c9c1be936f54: done: 0.017679 sec","job_status":"done","queue_duration_s":2.061248,"scheduling_latency_s":2.061265,"redis_calls":13,"redis_duration_s":0.002991,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002043,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017679,"completed_at":"2026-06-29T12:00:23.789Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000939,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1567vm-test-run-nixbot-gitlab> gitlab # [  235.750733] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"51e433d7f6c2bd4fd0d46f28","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-51e433d7f6c2bd4fd0d46f28: start","job_status":"start","queue_duration_s":2.080346,"scheduling_latency_s":2.080363}1568vm-test-run-nixbot-gitlab> gitlab # [  235.764511] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"51e433d7f6c2bd4fd0d46f28","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-51e433d7f6c2bd4fd0d46f28: done: 0.01648 sec","job_status":"done","queue_duration_s":2.080346,"scheduling_latency_s":2.080363,"redis_calls":13,"redis_duration_s":0.002787,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001853,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01648,"completed_at":"2026-06-29T12:00:23.807Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000835,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1569vm-test-run-nixbot-gitlab> gitlab # [  235.804893] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.808Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"d0e3e62b1eba30121afd429d","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-d0e3e62b1eba30121afd429d: start","job_status":"start","queue_duration_s":2.098267,"scheduling_latency_s":2.098285}1570vm-test-run-nixbot-gitlab> gitlab # [  235.819212] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.825Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"d0e3e62b1eba30121afd429d","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-d0e3e62b1eba30121afd429d: done: 0.01677 sec","job_status":"done","queue_duration_s":2.098267,"scheduling_latency_s":2.098285,"redis_calls":13,"redis_duration_s":0.002863,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001948,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01677,"completed_at":"2026-06-29T12:00:23.825Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000805,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1571vm-test-run-nixbot-gitlab> gitlab # [  235.859421] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"9e30210751fdd16422d728dc","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-9e30210751fdd16422d728dc: start","job_status":"start","queue_duration_s":2.116419,"scheduling_latency_s":2.116436}1572vm-test-run-nixbot-gitlab> gitlab # [  235.873140] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"9e30210751fdd16422d728dc","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-9e30210751fdd16422d728dc: done: 0.016822 sec","job_status":"done","queue_duration_s":2.116419,"scheduling_latency_s":2.116436,"redis_calls":13,"redis_duration_s":0.002883,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001912,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010414,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016822,"completed_at":"2026-06-29T12:00:23.843Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000858,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1573vm-test-run-nixbot-gitlab> gitlab # [  235.912964] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.845Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"0d3255e5d1aeda6f429179c9","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-0d3255e5d1aeda6f429179c9: start","job_status":"start","queue_duration_s":2.134704,"scheduling_latency_s":2.134721}1574vm-test-run-nixbot-gitlab> gitlab # [  235.926482] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"0d3255e5d1aeda6f429179c9","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-0d3255e5d1aeda6f429179c9: done: 0.016446 sec","job_status":"done","queue_duration_s":2.134704,"scheduling_latency_s":2.134721,"redis_calls":13,"redis_duration_s":0.0028409999999999998,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001887,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016446,"completed_at":"2026-06-29T12:00:23.861Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00076,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1575vm-test-run-nixbot-gitlab> gitlab # [  235.966797] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.863Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"d178ae500b9a4412cd3cff85","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-d178ae500b9a4412cd3cff85: start","job_status":"start","queue_duration_s":2.153134,"scheduling_latency_s":2.153152}1576vm-test-run-nixbot-gitlab> gitlab # [  235.980328] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.880Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"d178ae500b9a4412cd3cff85","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-d178ae500b9a4412cd3cff85: done: 0.0169 sec","job_status":"done","queue_duration_s":2.153134,"scheduling_latency_s":2.153152,"redis_calls":13,"redis_duration_s":0.003661,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000941,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00272,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0169,"completed_at":"2026-06-29T12:00:23.880Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000522,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1577vm-test-run-nixbot-gitlab> gitlab # [  236.021427] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.881Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"f4f856588366b2eaf1680687","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-f4f856588366b2eaf1680687: start","job_status":"start","queue_duration_s":2.17149,"scheduling_latency_s":2.171508}1578vm-test-run-nixbot-gitlab> gitlab # [  236.034733] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"f4f856588366b2eaf1680687","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-f4f856588366b2eaf1680687: done: 0.016304 sec","job_status":"done","queue_duration_s":2.17149,"scheduling_latency_s":2.171508,"redis_calls":13,"redis_duration_s":0.002811,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016304,"completed_at":"2026-06-29T12:00:23.898Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000493,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1579vm-test-run-nixbot-gitlab> gitlab # [  236.074998] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.899Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"f33b7aa5bfa2bfe8b595d880","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-f33b7aa5bfa2bfe8b595d880: start","job_status":"start","queue_duration_s":2.189116,"scheduling_latency_s":2.189132}1580vm-test-run-nixbot-gitlab> gitlab # [  236.088618] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"f33b7aa5bfa2bfe8b595d880","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-f33b7aa5bfa2bfe8b595d880: done: 0.016354 sec","job_status":"done","queue_duration_s":2.189116,"scheduling_latency_s":2.189132,"redis_calls":13,"redis_duration_s":0.002886,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000953,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016354,"completed_at":"2026-06-29T12:00:23.915Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000504,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1581vm-test-run-nixbot-gitlab> gitlab # [  236.129021] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c416a4a53a7e0dcdc83d0b48","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c416a4a53a7e0dcdc83d0b48: start","job_status":"start","queue_duration_s":2.207515,"scheduling_latency_s":2.207533}1582vm-test-run-nixbot-gitlab> gitlab # [  236.142391] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c416a4a53a7e0dcdc83d0b48","created_at":"2026-06-29T12:00:21.691Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-a9b2c2bdb03e4f75","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"79c6d7efd680ff9e4151c8ecb63b7894","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c416a4a53a7e0dcdc83d0b48: done: 0.016294 sec","job_status":"done","queue_duration_s":2.207515,"scheduling_latency_s":2.207533,"redis_calls":13,"redis_duration_s":0.0028060000000000003,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016294,"completed_at":"2026-06-29T12:00:23.934Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000539,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1583vm-test-run-nixbot-gitlab> gitlab # [  236.182800] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.935Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bcee5f66622c7e2453fe34b1","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bcee5f66622c7e2453fe34b1: start","job_status":"start","queue_duration_s":2.101828,"scheduling_latency_s":2.101844}1584vm-test-run-nixbot-gitlab> gitlab # [  236.195441] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.965Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bcee5f66622c7e2453fe34b1","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bcee5f66622c7e2453fe34b1: done: 0.02985 sec","job_status":"done","queue_duration_s":2.101828,"scheduling_latency_s":2.101844,"redis_calls":13,"redis_duration_s":0.003103,"redis_read_bytes":12,"redis_write_bytes":1300,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000454,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":289,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001166,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":395,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001483,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":616,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033724,"duration_s":0.02985,"completed_at":"2026-06-29T12:00:23.965Z","load_balancing_strategy":"primary","db_duration_s":0.00361,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1585vm-test-run-nixbot-gitlab> gitlab # [  236.237307] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.967Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"54d8326a7a0e7620300f0ea5","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-54d8326a7a0e7620300f0ea5: start","job_status":"start","queue_duration_s":2.133588,"scheduling_latency_s":2.133606}1586vm-test-run-nixbot-gitlab> gitlab # [  236.250134] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.985Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"54d8326a7a0e7620300f0ea5","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-54d8326a7a0e7620300f0ea5: done: 0.018327 sec","job_status":"done","queue_duration_s":2.133588,"scheduling_latency_s":2.133606,"redis_calls":10,"redis_duration_s":0.002706,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000932,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001774,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011961,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018327,"completed_at":"2026-06-29T12:00:23.985Z","load_balancing_strategy":"primary","db_duration_s":0.001212,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1587vm-test-run-nixbot-gitlab> gitlab # [  236.289414] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:23.986Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9cab4e679d0f894e7eb5125e","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9cab4e679d0f894e7eb5125e: start","job_status":"start","queue_duration_s":2.153229,"scheduling_latency_s":2.153246}1588vm-test-run-nixbot-gitlab> gitlab # [  236.301718] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.005Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9cab4e679d0f894e7eb5125e","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9cab4e679d0f894e7eb5125e: done: 0.018326 sec","job_status":"done","queue_duration_s":2.153229,"scheduling_latency_s":2.153246,"redis_calls":10,"redis_duration_s":0.002556,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00101,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001546,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018326,"completed_at":"2026-06-29T12:00:24.005Z","load_balancing_strategy":"primary","db_duration_s":0.001266,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1589vm-test-run-nixbot-gitlab> gitlab # [  236.340697] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.006Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d1142c0c35e6a27932c7d020","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d1142c0c35e6a27932c7d020: start","job_status":"start","queue_duration_s":2.172932,"scheduling_latency_s":2.172951}1590vm-test-run-nixbot-gitlab> gitlab # [  236.353185] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d1142c0c35e6a27932c7d020","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d1142c0c35e6a27932c7d020: done: 0.018606 sec","job_status":"done","queue_duration_s":2.172932,"scheduling_latency_s":2.172951,"redis_calls":10,"redis_duration_s":0.0022900000000000004,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001347,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018606,"completed_at":"2026-06-29T12:00:24.025Z","load_balancing_strategy":"primary","db_duration_s":0.001185,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1591vm-test-run-nixbot-gitlab> gitlab # [  236.391923] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.026Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"14dc7898b28a0e19246ec0fd","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-14dc7898b28a0e19246ec0fd: start","job_status":"start","queue_duration_s":2.192856,"scheduling_latency_s":2.192873}1592vm-test-run-nixbot-gitlab> gitlab # [  236.404484] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.044Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"14dc7898b28a0e19246ec0fd","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-14dc7898b28a0e19246ec0fd: done: 0.018268 sec","job_status":"done","queue_duration_s":2.192856,"scheduling_latency_s":2.192873,"redis_calls":10,"redis_duration_s":0.002472,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001035,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001437,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018268,"completed_at":"2026-06-29T12:00:24.044Z","load_balancing_strategy":"primary","db_duration_s":0.001319,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1593vm-test-run-nixbot-gitlab> gitlab # [  236.444364] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.046Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f32c9c1f43f61290db1cf4d9","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f32c9c1f43f61290db1cf4d9: start","job_status":"start","queue_duration_s":2.21249,"scheduling_latency_s":2.212508}1594vm-test-run-nixbot-gitlab> gitlab # [  236.456386] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f32c9c1f43f61290db1cf4d9","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f32c9c1f43f61290db1cf4d9: done: 0.019526 sec","job_status":"done","queue_duration_s":2.21249,"scheduling_latency_s":2.212508,"redis_calls":10,"redis_duration_s":0.002445,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001022,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001423,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011516,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019526,"completed_at":"2026-06-29T12:00:24.065Z","load_balancing_strategy":"primary","db_duration_s":0.002452,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1595vm-test-run-nixbot-gitlab> gitlab # [  236.494840] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"01a80245360facd1b5fb5562","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-01a80245360facd1b5fb5562: start","job_status":"start","queue_duration_s":2.233389,"scheduling_latency_s":2.233407}1596vm-test-run-nixbot-gitlab> gitlab # [  236.507396] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.086Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"01a80245360facd1b5fb5562","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-01a80245360facd1b5fb5562: done: 0.019164 sec","job_status":"done","queue_duration_s":2.233389,"scheduling_latency_s":2.233407,"redis_calls":10,"redis_duration_s":0.0025919999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001076,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001516,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019164,"completed_at":"2026-06-29T12:00:24.086Z","load_balancing_strategy":"primary","db_duration_s":0.003106,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1597vm-test-run-nixbot-gitlab> gitlab # [  236.546116] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.087Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c95e80d3b102e7ce90612147","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c95e80d3b102e7ce90612147: start","job_status":"start","queue_duration_s":2.253947,"scheduling_latency_s":2.253965}1598vm-test-run-nixbot-gitlab> gitlab # [  236.558450] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.106Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c95e80d3b102e7ce90612147","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c95e80d3b102e7ce90612147: done: 0.018988 sec","job_status":"done","queue_duration_s":2.253947,"scheduling_latency_s":2.253965,"redis_calls":10,"redis_duration_s":0.002533,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001038,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001495,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018988,"completed_at":"2026-06-29T12:00:24.106Z","load_balancing_strategy":"primary","db_duration_s":0.002303,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1599vm-test-run-nixbot-gitlab> gitlab # [  236.597231] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.108Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e0dd82e0e7b2754c0c0a9fd8","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e0dd82e0e7b2754c0c0a9fd8: start","job_status":"start","queue_duration_s":2.27436,"scheduling_latency_s":2.274378}1600vm-test-run-nixbot-gitlab> gitlab # [  236.609275] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e0dd82e0e7b2754c0c0a9fd8","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e0dd82e0e7b2754c0c0a9fd8: done: 0.018665 sec","job_status":"done","queue_duration_s":2.27436,"scheduling_latency_s":2.274378,"redis_calls":10,"redis_duration_s":0.002493,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001036,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001457,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011695,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018665,"completed_at":"2026-06-29T12:00:24.126Z","load_balancing_strategy":"primary","db_duration_s":0.002273,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1601vm-test-run-nixbot-gitlab> gitlab # [  236.649235] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.128Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"37f2f4e8a47d727e5db6103a","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-37f2f4e8a47d727e5db6103a: start","job_status":"start","queue_duration_s":2.294476,"scheduling_latency_s":2.294494}1602vm-test-run-nixbot-gitlab> gitlab # [  236.661234] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"37f2f4e8a47d727e5db6103a","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-37f2f4e8a47d727e5db6103a: done: 0.018796 sec","job_status":"done","queue_duration_s":2.294476,"scheduling_latency_s":2.294494,"redis_calls":10,"redis_duration_s":0.002503,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001042,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001461,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011659,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018796,"completed_at":"2026-06-29T12:00:24.146Z","load_balancing_strategy":"primary","db_duration_s":0.002412,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1603vm-test-run-nixbot-gitlab> gitlab # [  236.699646] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2a04ef6ecd4fbb68753d9cd1","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2a04ef6ecd4fbb68753d9cd1: start","job_status":"start","queue_duration_s":2.314457,"scheduling_latency_s":2.314475}1604vm-test-run-nixbot-gitlab> gitlab # [  236.712106] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2a04ef6ecd4fbb68753d9cd1","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2a04ef6ecd4fbb68753d9cd1: done: 0.019317 sec","job_status":"done","queue_duration_s":2.314457,"scheduling_latency_s":2.314475,"redis_calls":10,"redis_duration_s":0.002413,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001003,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00141,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011909,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019317,"completed_at":"2026-06-29T12:00:24.167Z","load_balancing_strategy":"primary","db_duration_s":0.002344,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1605vm-test-run-nixbot-gitlab> gitlab # [  236.750476] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a6418386921ce1a16d90807b","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a6418386921ce1a16d90807b: start","job_status":"start","queue_duration_s":2.335201,"scheduling_latency_s":2.335218}1606vm-test-run-nixbot-gitlab> gitlab # [  236.762587] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a6418386921ce1a16d90807b","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a6418386921ce1a16d90807b: done: 0.019057 sec","job_status":"done","queue_duration_s":2.335201,"scheduling_latency_s":2.335218,"redis_calls":10,"redis_duration_s":0.002604,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001072,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001532,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011689,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019057,"completed_at":"2026-06-29T12:00:24.187Z","load_balancing_strategy":"primary","db_duration_s":0.002889,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1607vm-test-run-nixbot-gitlab> gitlab # [  236.801319] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.189Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"38e99c43dbb4d4f7b06bacc1","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-38e99c43dbb4d4f7b06bacc1: start","job_status":"start","queue_duration_s":2.355555,"scheduling_latency_s":2.355572}1608vm-test-run-nixbot-gitlab> gitlab # [  236.813713] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.207Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"38e99c43dbb4d4f7b06bacc1","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-38e99c43dbb4d4f7b06bacc1: done: 0.018661 sec","job_status":"done","queue_duration_s":2.355555,"scheduling_latency_s":2.355572,"redis_calls":10,"redis_duration_s":0.0024549999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001037,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001418,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011603,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018661,"completed_at":"2026-06-29T12:00:24.207Z","load_balancing_strategy":"primary","db_duration_s":0.002352,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1609vm-test-run-nixbot-gitlab> gitlab # [  236.853516] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.209Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2448b6fb7136a871f43ab4c6","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2448b6fb7136a871f43ab4c6: start","job_status":"start","queue_duration_s":2.375774,"scheduling_latency_s":2.375792}1610vm-test-run-nixbot-gitlab> gitlab # [  236.865811] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.227Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2448b6fb7136a871f43ab4c6","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2448b6fb7136a871f43ab4c6: done: 0.018504 sec","job_status":"done","queue_duration_s":2.375774,"scheduling_latency_s":2.375792,"redis_calls":10,"redis_duration_s":0.002379,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001287,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011552,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018504,"completed_at":"2026-06-29T12:00:24.227Z","load_balancing_strategy":"primary","db_duration_s":0.002309,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1611vm-test-run-nixbot-gitlab> gitlab # [  236.904125] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e7949fa74d74abdaede376d6","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e7949fa74d74abdaede376d6: start","job_status":"start","queue_duration_s":2.395594,"scheduling_latency_s":2.395612}1612vm-test-run-nixbot-gitlab> gitlab # [  236.916408] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.247Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e7949fa74d74abdaede376d6","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e7949fa74d74abdaede376d6: done: 0.018585 sec","job_status":"done","queue_duration_s":2.395594,"scheduling_latency_s":2.395612,"redis_calls":10,"redis_duration_s":0.0024279999999999996,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001022,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001406,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018585,"completed_at":"2026-06-29T12:00:24.247Z","load_balancing_strategy":"primary","db_duration_s":0.002353,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1613vm-test-run-nixbot-gitlab> gitlab # [  236.955678] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.249Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ce2839821dcd7135e145f861","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ce2839821dcd7135e145f861: start","job_status":"start","queue_duration_s":2.415561,"scheduling_latency_s":2.41558}1614vm-test-run-nixbot-gitlab> gitlab # [  236.968421] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.268Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ce2839821dcd7135e145f861","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ce2839821dcd7135e145f861: done: 0.018967 sec","job_status":"done","queue_duration_s":2.415561,"scheduling_latency_s":2.41558,"redis_calls":10,"redis_duration_s":0.0025900000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001497,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011603,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018967,"completed_at":"2026-06-29T12:00:24.268Z","load_balancing_strategy":"primary","db_duration_s":0.002525,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1615vm-test-run-nixbot-gitlab> gitlab # [  237.007627] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"847524a7dbdb53373b491db9","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-847524a7dbdb53373b491db9: start","job_status":"start","queue_duration_s":2.436209,"scheduling_latency_s":2.436227}1616vm-test-run-nixbot-gitlab> gitlab # [  237.020204] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"847524a7dbdb53373b491db9","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-847524a7dbdb53373b491db9: done: 0.018849 sec","job_status":"done","queue_duration_s":2.436209,"scheduling_latency_s":2.436227,"redis_calls":10,"redis_duration_s":0.0024600000000000004,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00102,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00144,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018849,"completed_at":"2026-06-29T12:00:24.288Z","load_balancing_strategy":"primary","db_duration_s":0.002918,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1617vm-test-run-nixbot-gitlab> gitlab # [  237.060131] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.290Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"917d2e5abeed69785ee72c5f","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-917d2e5abeed69785ee72c5f: start","job_status":"start","queue_duration_s":2.456466,"scheduling_latency_s":2.456484}1618vm-test-run-nixbot-gitlab> gitlab # [  237.072336] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"917d2e5abeed69785ee72c5f","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-917d2e5abeed69785ee72c5f: done: 0.018767 sec","job_status":"done","queue_duration_s":2.456466,"scheduling_latency_s":2.456484,"redis_calls":10,"redis_duration_s":0.002405,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001059,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001346,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018767,"completed_at":"2026-06-29T12:00:24.308Z","load_balancing_strategy":"primary","db_duration_s":0.002354,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1619vm-test-run-nixbot-gitlab> gitlab # [  237.111605] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.310Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cdd2a1995f36918b37f8e76e","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cdd2a1995f36918b37f8e76e: start","job_status":"start","queue_duration_s":2.476699,"scheduling_latency_s":2.476718}1620vm-test-run-nixbot-gitlab> gitlab # [  237.124226] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.329Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cdd2a1995f36918b37f8e76e","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cdd2a1995f36918b37f8e76e: done: 0.018904 sec","job_status":"done","queue_duration_s":2.476699,"scheduling_latency_s":2.476718,"redis_calls":10,"redis_duration_s":0.002411,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000997,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001414,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018904,"completed_at":"2026-06-29T12:00:24.329Z","load_balancing_strategy":"primary","db_duration_s":0.002402,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1621vm-test-run-nixbot-gitlab> gitlab # [  237.162500] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0b83c15808fd8bb717ae7bb6","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0b83c15808fd8bb717ae7bb6: start","job_status":"start","queue_duration_s":2.496957,"scheduling_latency_s":2.496974}1622vm-test-run-nixbot-gitlab> gitlab # [  237.174861] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.354Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0b83c15808fd8bb717ae7bb6","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0b83c15808fd8bb717ae7bb6: done: 0.023497 sec","job_status":"done","queue_duration_s":2.496957,"scheduling_latency_s":2.496974,"redis_calls":10,"redis_duration_s":0.001687,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000707,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00098,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.006,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023497,"completed_at":"2026-06-29T12:00:24.354Z","load_balancing_strategy":"primary","db_duration_s":0.00696,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1623vm-test-run-nixbot-gitlab> gitlab # [  237.213470] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d3e148a0d1af28a761bb33b3","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d3e148a0d1af28a761bb33b3: start","job_status":"start","queue_duration_s":2.521574,"scheduling_latency_s":2.521593}1624vm-test-run-nixbot-gitlab> gitlab # [  237.225526] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.372Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d3e148a0d1af28a761bb33b3","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d3e148a0d1af28a761bb33b3: done: 0.017646 sec","job_status":"done","queue_duration_s":2.521574,"scheduling_latency_s":2.521593,"redis_calls":10,"redis_duration_s":0.001483,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000576,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012399,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017646,"completed_at":"2026-06-29T12:00:24.372Z","load_balancing_strategy":"primary","db_duration_s":0.00163,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1625vm-test-run-nixbot-gitlab> gitlab # [  237.265309] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"655b19e77c48e75240aac7f1","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-655b19e77c48e75240aac7f1: start","job_status":"start","queue_duration_s":2.540347,"scheduling_latency_s":2.540366}1626vm-test-run-nixbot-gitlab> gitlab # [  237.277749] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"655b19e77c48e75240aac7f1","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-655b19e77c48e75240aac7f1: done: 0.017398 sec","job_status":"done","queue_duration_s":2.540347,"scheduling_latency_s":2.540366,"redis_calls":10,"redis_duration_s":0.001598,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000809,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000789,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011701,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017398,"completed_at":"2026-06-29T12:00:24.391Z","load_balancing_strategy":"primary","db_duration_s":0.001288,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1627vm-test-run-nixbot-gitlab> gitlab # [  237.316504] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.392Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3b83ec70cd4052126d67adb2","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3b83ec70cd4052126d67adb2: start","job_status":"start","queue_duration_s":2.558901,"scheduling_latency_s":2.55892}1628vm-test-run-nixbot-gitlab> gitlab # [  237.328548] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.409Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3b83ec70cd4052126d67adb2","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3b83ec70cd4052126d67adb2: done: 0.017116 sec","job_status":"done","queue_duration_s":2.558901,"scheduling_latency_s":2.55892,"redis_calls":10,"redis_duration_s":0.001407,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000567,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00084,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011969,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017116,"completed_at":"2026-06-29T12:00:24.409Z","load_balancing_strategy":"primary","db_duration_s":0.001397,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1629vm-test-run-nixbot-gitlab> gitlab # [  237.367161] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b17e919f002fbb1a0d541ace","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b17e919f002fbb1a0d541ace: start","job_status":"start","queue_duration_s":2.577176,"scheduling_latency_s":2.577195}1630vm-test-run-nixbot-gitlab> gitlab # [  237.379623] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b17e919f002fbb1a0d541ace","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b17e919f002fbb1a0d541ace: done: 0.017289 sec","job_status":"done","queue_duration_s":2.577176,"scheduling_latency_s":2.577195,"redis_calls":10,"redis_duration_s":0.001541,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00075,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017289,"completed_at":"2026-06-29T12:00:24.428Z","load_balancing_strategy":"primary","db_duration_s":0.001204,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1631vm-test-run-nixbot-gitlab> gitlab # [  237.418128] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7cb431a1b683f4697eb8df51","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7cb431a1b683f4697eb8df51: start","job_status":"start","queue_duration_s":2.595613,"scheduling_latency_s":2.595632}1632vm-test-run-nixbot-gitlab> gitlab # [  237.430529] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.446Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7cb431a1b683f4697eb8df51","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7cb431a1b683f4697eb8df51: done: 0.016932 sec","job_status":"done","queue_duration_s":2.595613,"scheduling_latency_s":2.595632,"redis_calls":10,"redis_duration_s":0.001401,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000827,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016932,"completed_at":"2026-06-29T12:00:24.446Z","load_balancing_strategy":"primary","db_duration_s":0.001242,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1633vm-test-run-nixbot-gitlab> gitlab # [  237.470210] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.447Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5e0b10718d13f0f1b88ba1ba","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5e0b10718d13f0f1b88ba1ba: start","job_status":"start","queue_duration_s":2.61403,"scheduling_latency_s":2.614049}1634vm-test-run-nixbot-gitlab> gitlab # [  237.482335] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5e0b10718d13f0f1b88ba1ba","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5e0b10718d13f0f1b88ba1ba: done: 0.017001 sec","job_status":"done","queue_duration_s":2.61403,"scheduling_latency_s":2.614049,"redis_calls":10,"redis_duration_s":0.001338,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000584,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000754,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017001,"completed_at":"2026-06-29T12:00:24.464Z","load_balancing_strategy":"primary","db_duration_s":0.00121,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1635vm-test-run-nixbot-gitlab> gitlab # [  237.520826] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4b22b4094177babdf93b605c","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4b22b4094177babdf93b605c: start","job_status":"start","queue_duration_s":2.632148,"scheduling_latency_s":2.632167}1636vm-test-run-nixbot-gitlab> gitlab # [  237.533065] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.483Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4b22b4094177babdf93b605c","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4b22b4094177babdf93b605c: done: 0.017752 sec","job_status":"done","queue_duration_s":2.632148,"scheduling_latency_s":2.632167,"redis_calls":10,"redis_duration_s":0.0016419999999999998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000602,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00104,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012553,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017752,"completed_at":"2026-06-29T12:00:24.483Z","load_balancing_strategy":"primary","db_duration_s":0.001233,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1637vm-test-run-nixbot-gitlab> gitlab # [  237.571923] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5d8c365aea96fc2a3f83d340","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5d8c365aea96fc2a3f83d340: start","job_status":"start","queue_duration_s":2.651111,"scheduling_latency_s":2.65113}1638vm-test-run-nixbot-gitlab> gitlab # [  237.584262] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.501Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5d8c365aea96fc2a3f83d340","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5d8c365aea96fc2a3f83d340: done: 0.016642 sec","job_status":"done","queue_duration_s":2.651111,"scheduling_latency_s":2.65113,"redis_calls":10,"redis_duration_s":0.001325,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000761,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016642,"completed_at":"2026-06-29T12:00:24.501Z","load_balancing_strategy":"primary","db_duration_s":0.001184,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1639vm-test-run-nixbot-gitlab> gitlab # [  237.622853] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.502Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e0d13bcd66fdf9897d09b1d1","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e0d13bcd66fdf9897d09b1d1: start","job_status":"start","queue_duration_s":2.669045,"scheduling_latency_s":2.669065}1640vm-test-run-nixbot-gitlab> gitlab # [  237.635367] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.519Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e0d13bcd66fdf9897d09b1d1","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e0d13bcd66fdf9897d09b1d1: done: 0.017146 sec","job_status":"done","queue_duration_s":2.669045,"scheduling_latency_s":2.669065,"redis_calls":10,"redis_duration_s":0.001337,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000589,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000748,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017146,"completed_at":"2026-06-29T12:00:24.519Z","load_balancing_strategy":"primary","db_duration_s":0.001185,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1641vm-test-run-nixbot-gitlab> gitlab # [  237.675481] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5dc8d9fc32f336393b466fef","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5dc8d9fc32f336393b466fef: start","job_status":"start","queue_duration_s":2.687406,"scheduling_latency_s":2.687426}1642vm-test-run-nixbot-gitlab> gitlab # [  237.688180] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5dc8d9fc32f336393b466fef","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5dc8d9fc32f336393b466fef: done: 0.017095 sec","job_status":"done","queue_duration_s":2.687406,"scheduling_latency_s":2.687426,"redis_calls":10,"redis_duration_s":0.001388,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000576,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000812,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017095,"completed_at":"2026-06-29T12:00:24.538Z","load_balancing_strategy":"primary","db_duration_s":0.001187,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1643vm-test-run-nixbot-gitlab> gitlab # [  237.726676] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.539Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"19c8d23c36759c8a578079aa","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-19c8d23c36759c8a578079aa: start","job_status":"start","queue_duration_s":2.705629,"scheduling_latency_s":2.705648}1644vm-test-run-nixbot-gitlab> gitlab # [  237.739142] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"19c8d23c36759c8a578079aa","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-19c8d23c36759c8a578079aa: done: 0.016891 sec","job_status":"done","queue_duration_s":2.705629,"scheduling_latency_s":2.705648,"redis_calls":10,"redis_duration_s":0.001316,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000759,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011669,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016891,"completed_at":"2026-06-29T12:00:24.556Z","load_balancing_strategy":"primary","db_duration_s":0.001204,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1645vm-test-run-nixbot-gitlab> gitlab # [  237.777602] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.557Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8c892b0228031dd4308f7254","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8c892b0228031dd4308f7254: start","job_status":"start","queue_duration_s":2.724053,"scheduling_latency_s":2.724073}1646vm-test-run-nixbot-gitlab> gitlab # [  237.789851] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.575Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8c892b0228031dd4308f7254","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8c892b0228031dd4308f7254: done: 0.017401 sec","job_status":"done","queue_duration_s":2.724053,"scheduling_latency_s":2.724073,"redis_calls":10,"redis_duration_s":0.00139,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000815,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017401,"completed_at":"2026-06-29T12:00:24.575Z","load_balancing_strategy":"primary","db_duration_s":0.001283,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1647vm-test-run-nixbot-gitlab> gitlab # [  237.829174] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.576Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"90316a237f72a242af51f78b","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-90316a237f72a242af51f78b: start","job_status":"start","queue_duration_s":2.74256,"scheduling_latency_s":2.742578}1648vm-test-run-nixbot-gitlab> gitlab # [  237.841589] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"90316a237f72a242af51f78b","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-90316a237f72a242af51f78b: done: 0.016997 sec","job_status":"done","queue_duration_s":2.74256,"scheduling_latency_s":2.742578,"redis_calls":10,"redis_duration_s":0.001484,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016997,"completed_at":"2026-06-29T12:00:24.593Z","load_balancing_strategy":"primary","db_duration_s":0.001169,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1649vm-test-run-nixbot-gitlab> gitlab # [  237.881625] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"540e8a729b769295418f0fd7","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-540e8a729b769295418f0fd7: start","job_status":"start","queue_duration_s":2.760681,"scheduling_latency_s":2.7607}1650vm-test-run-nixbot-gitlab> gitlab # [  237.894233] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"540e8a729b769295418f0fd7","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-540e8a729b769295418f0fd7: done: 0.017117 sec","job_status":"done","queue_duration_s":2.760681,"scheduling_latency_s":2.7607,"redis_calls":10,"redis_duration_s":0.001428,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000627,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000801,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011851,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017117,"completed_at":"2026-06-29T12:00:24.611Z","load_balancing_strategy":"primary","db_duration_s":0.001297,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1651vm-test-run-nixbot-gitlab> gitlab # [  237.932486] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.613Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1cc06d3c084fd42a0127e4e0","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1cc06d3c084fd42a0127e4e0: start","job_status":"start","queue_duration_s":2.77927,"scheduling_latency_s":2.77929}1652vm-test-run-nixbot-gitlab> gitlab # [  237.944774] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.629Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1cc06d3c084fd42a0127e4e0","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1cc06d3c084fd42a0127e4e0: done: 0.01698 sec","job_status":"done","queue_duration_s":2.77927,"scheduling_latency_s":2.77929,"redis_calls":10,"redis_duration_s":0.0013289999999999999,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000773,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01698,"completed_at":"2026-06-29T12:00:24.629Z","load_balancing_strategy":"primary","db_duration_s":0.001189,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1653vm-test-run-nixbot-gitlab> gitlab # [  237.983431] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.631Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4e17cac6655b5c69cfc4de37","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4e17cac6655b5c69cfc4de37: start","job_status":"start","queue_duration_s":2.797385,"scheduling_latency_s":2.797403}1654vm-test-run-nixbot-gitlab> gitlab # [  237.995547] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4e17cac6655b5c69cfc4de37","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4e17cac6655b5c69cfc4de37: done: 0.016834 sec","job_status":"done","queue_duration_s":2.797385,"scheduling_latency_s":2.797403,"redis_calls":10,"redis_duration_s":0.0015990000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000845,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000754,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016834,"completed_at":"2026-06-29T12:00:24.647Z","load_balancing_strategy":"primary","db_duration_s":0.001199,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1655vm-test-run-nixbot-gitlab> gitlab # [  238.034532] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"16427ee54e6db8f4a1c49f5c","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-16427ee54e6db8f4a1c49f5c: start","job_status":"start","queue_duration_s":2.815425,"scheduling_latency_s":2.815444}1656vm-test-run-nixbot-gitlab> gitlab # [  238.046984] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.666Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"16427ee54e6db8f4a1c49f5c","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-16427ee54e6db8f4a1c49f5c: done: 0.017076 sec","job_status":"done","queue_duration_s":2.815425,"scheduling_latency_s":2.815444,"redis_calls":10,"redis_duration_s":0.0013830000000000001,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000817,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011771,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017076,"completed_at":"2026-06-29T12:00:24.666Z","load_balancing_strategy":"primary","db_duration_s":0.00119,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1657vm-test-run-nixbot-gitlab> gitlab # [  238.087203] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"233af95c7a9f612c662ba731","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-233af95c7a9f612c662ba731: start","job_status":"start","queue_duration_s":2.833878,"scheduling_latency_s":2.833898}1658vm-test-run-nixbot-gitlab> gitlab # [  238.099595] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.684Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"233af95c7a9f612c662ba731","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-233af95c7a9f612c662ba731: done: 0.0172 sec","job_status":"done","queue_duration_s":2.833878,"scheduling_latency_s":2.833898,"redis_calls":10,"redis_duration_s":0.0014680000000000001,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000706,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000762,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0172,"completed_at":"2026-06-29T12:00:24.684Z","load_balancing_strategy":"primary","db_duration_s":0.001216,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1659vm-test-run-nixbot-gitlab> gitlab # [  238.138705] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.685Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9b50c33c42cb64812b2764ab","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9b50c33c42cb64812b2764ab: start","job_status":"start","queue_duration_s":2.852192,"scheduling_latency_s":2.852211}1660vm-test-run-nixbot-gitlab> gitlab # [  238.151184] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.703Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9b50c33c42cb64812b2764ab","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9b50c33c42cb64812b2764ab: done: 0.017136 sec","job_status":"done","queue_duration_s":2.852192,"scheduling_latency_s":2.852211,"redis_calls":10,"redis_duration_s":0.0014069999999999998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000614,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000793,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017136,"completed_at":"2026-06-29T12:00:24.702Z","load_balancing_strategy":"primary","db_duration_s":0.001223,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1661vm-test-run-nixbot-gitlab> gitlab # [  238.190076] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ae20d18a90d29a27742c9d19","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ae20d18a90d29a27742c9d19: start","job_status":"start","queue_duration_s":2.870403,"scheduling_latency_s":2.870423}1662vm-test-run-nixbot-gitlab> gitlab # [  238.202365] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ae20d18a90d29a27742c9d19","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ae20d18a90d29a27742c9d19: done: 0.016783 sec","job_status":"done","queue_duration_s":2.870403,"scheduling_latency_s":2.870423,"redis_calls":10,"redis_duration_s":0.001315,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000759,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016783,"completed_at":"2026-06-29T12:00:24.720Z","load_balancing_strategy":"primary","db_duration_s":0.0012,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1663vm-test-run-nixbot-gitlab> gitlab # [  238.241019] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5d1dea449969437b95445a33","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5d1dea449969437b95445a33: start","job_status":"start","queue_duration_s":2.88848,"scheduling_latency_s":2.888499}1664vm-test-run-nixbot-gitlab> gitlab # [  238.253405] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.739Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5d1dea449969437b95445a33","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5d1dea449969437b95445a33: done: 0.017046 sec","job_status":"done","queue_duration_s":2.88848,"scheduling_latency_s":2.888499,"redis_calls":10,"redis_duration_s":0.001448,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000625,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000823,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017046,"completed_at":"2026-06-29T12:00:24.739Z","load_balancing_strategy":"primary","db_duration_s":0.001239,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1665vm-test-run-nixbot-gitlab> gitlab # [  238.293601] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.740Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3b9368ecbc1ad525d8ea8523","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3b9368ecbc1ad525d8ea8523: start","job_status":"start","queue_duration_s":2.906674,"scheduling_latency_s":2.906694}1666vm-test-run-nixbot-gitlab> gitlab # [  238.305898] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3b9368ecbc1ad525d8ea8523","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3b9368ecbc1ad525d8ea8523: done: 0.016901 sec","job_status":"done","queue_duration_s":2.906674,"scheduling_latency_s":2.906694,"redis_calls":10,"redis_duration_s":0.0013419999999999999,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000776,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016901,"completed_at":"2026-06-29T12:00:24.757Z","load_balancing_strategy":"primary","db_duration_s":0.001177,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1667vm-test-run-nixbot-gitlab> gitlab # [  238.344608] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a86c12819f3735e417dd7b1b","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a86c12819f3735e417dd7b1b: start","job_status":"start","queue_duration_s":2.924702,"scheduling_latency_s":2.924719}1668vm-test-run-nixbot-gitlab> gitlab # [  238.356869] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.944Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a86c12819f3735e417dd7b1b","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a86c12819f3735e417dd7b1b: done: 0.186059 sec","job_status":"done","queue_duration_s":2.924702,"scheduling_latency_s":2.924719,"redis_calls":10,"redis_duration_s":0.002285,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00151,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000775,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.186059,"completed_at":"2026-06-29T12:00:24.944Z","load_balancing_strategy":"primary","db_duration_s":0.001208,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1669vm-test-run-nixbot-gitlab> gitlab # [  238.395431] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.948Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"479d6cca25bed3cd469731b0","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-479d6cca25bed3cd469731b0: start","job_status":"start","queue_duration_s":3.114755,"scheduling_latency_s":3.114805}1670vm-test-run-nixbot-gitlab> gitlab # [  238.407900] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"479d6cca25bed3cd469731b0","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-479d6cca25bed3cd469731b0: done: 0.04495 sec","job_status":"done","queue_duration_s":3.114755,"scheduling_latency_s":3.114805,"redis_calls":10,"redis_duration_s":0.004111999999999999,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001956,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002156,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04495,"completed_at":"2026-06-29T12:00:24.993Z","load_balancing_strategy":"primary","db_duration_s":0.003508,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1671vm-test-run-nixbot-gitlab> gitlab # [  238.446333] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:24.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1a70080ad762cfdd05b9bd1f","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1a70080ad762cfdd05b9bd1f: start","job_status":"start","queue_duration_s":3.162472,"scheduling_latency_s":3.162513}1672vm-test-run-nixbot-gitlab> gitlab # [  238.458716] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:25.031Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1a70080ad762cfdd05b9bd1f","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1a70080ad762cfdd05b9bd1f: done: 0.034999 sec","job_status":"done","queue_duration_s":3.162472,"scheduling_latency_s":3.162513,"redis_calls":10,"redis_duration_s":0.0032389999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001413,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001826,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034999,"completed_at":"2026-06-29T12:00:25.031Z","load_balancing_strategy":"primary","db_duration_s":0.003023,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1673vm-test-run-nixbot-gitlab> gitlab # [  238.498201] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:25.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"918d9a21c3933094eb45f142","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-918d9a21c3933094eb45f142: start","job_status":"start","queue_duration_s":3.19973,"scheduling_latency_s":3.199763}1674vm-test-run-nixbot-gitlab> gitlab # [  238.510226] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:25.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"918d9a21c3933094eb45f142","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-918d9a21c3933094eb45f142: done: 0.032097 sec","job_status":"done","queue_duration_s":3.19973,"scheduling_latency_s":3.199763,"redis_calls":10,"redis_duration_s":0.003283,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001356,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001927,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032097,"completed_at":"2026-06-29T12:00:25.065Z","load_balancing_strategy":"primary","db_duration_s":0.002817,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1675vm-test-run-nixbot-gitlab> gitlab # [  238.548706] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:25.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"70720a1e6569fc821acaada1","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-70720a1e6569fc821acaada1: start","job_status":"start","queue_duration_s":3.234256,"scheduling_latency_s":3.234285}1676vm-test-run-nixbot-gitlab> gitlab # [  238.560957] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:25.099Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"70720a1e6569fc821acaada1","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-70720a1e6569fc821acaada1: done: 0.031334 sec","job_status":"done","queue_duration_s":3.234256,"scheduling_latency_s":3.234285,"redis_calls":10,"redis_duration_s":0.0033160000000000004,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001345,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001971,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031334,"completed_at":"2026-06-29T12:00:25.099Z","load_balancing_strategy":"primary","db_duration_s":0.003687,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1677vm-test-run-nixbot-gitlab> gitlab # [  238.599332] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:25.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fa80ec56b798a73780a4eb31","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fa80ec56b798a73780a4eb31: start","job_status":"start","queue_duration_s":3.267529,"scheduling_latency_s":3.267555}1678vm-test-run-nixbot-gitlab> gitlab # [  238.611864] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:25.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fa80ec56b798a73780a4eb31","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fa80ec56b798a73780a4eb31: done: 0.025148 sec","job_status":"done","queue_duration_s":3.267529,"scheduling_latency_s":3.267555,"redis_calls":10,"redis_duration_s":0.002822,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001202,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00162,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025148,"completed_at":"2026-06-29T12:00:25.126Z","load_balancing_strategy":"primary","db_duration_s":0.002825,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1679vm-test-run-nixbot-gitlab> gitlab # [  238.650490] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:25.128Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fc361f586a525030eaf3d09a","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fc361f586a525030eaf3d09a: start","job_status":"start","queue_duration_s":3.294339,"scheduling_latency_s":3.294362}1680vm-test-run-nixbot-gitlab> gitlab # [  238.662797] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:25.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fc361f586a525030eaf3d09a","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fc361f586a525030eaf3d09a: done: 0.022699 sec","job_status":"done","queue_duration_s":3.294339,"scheduling_latency_s":3.294362,"redis_calls":10,"redis_duration_s":0.002636,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001109,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001527,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022699,"completed_at":"2026-06-29T12:00:25.150Z","load_balancing_strategy":"primary","db_duration_s":0.002577,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1681vm-test-run-nixbot-gitlab> gitlab # [  238.701934] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:25.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7c4ae58dade81be84d6bb165","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7c4ae58dade81be84d6bb165: start","job_status":"start","queue_duration_s":3.318902,"scheduling_latency_s":3.318925}1682vm-test-run-nixbot-gitlab> gitlab # [  238.714337] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:00:25.174Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7c4ae58dade81be84d6bb165","created_at":"2026-06-29T12:00:21.759Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-5a6b80adcc5f45dd","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"002783f81532c1e96f9ebd1d53449432","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:00:21.833Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7c4ae58dade81be84d6bb165: done: 0.021711 sec","job_status":"done","queue_duration_s":3.318902,"scheduling_latency_s":3.318925,"redis_calls":10,"redis_duration_s":0.002666,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001134,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001532,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013774,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021711,"completed_at":"2026-06-29T12:00:25.174Z","load_balancing_strategy":"primary","db_duration_s":0.002572,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1683vm-test-run-nixbot-gitlab> gitlab # [  256.471441] setup-gitlab-start[3284]: W, [2026-06-29T12:00:46.334425 #3284]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1684vm-test-run-nixbot-gitlab> gitlab # [  275.555419] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:05.418Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"9163f405c42aa69851ae1644","created_at":"2026-06-29T12:01:05.405Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91c8caa73992901f320b69fd4ca00065","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T12:01:05.415Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-9163f405c42aa69851ae1644: start","job_status":"start","queue_duration_s":0.002455,"scheduling_latency_s":0.002493}1685vm-test-run-nixbot-gitlab> gitlab # [  275.777688] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:05.639Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"9163f405c42aa69851ae1644","created_at":"2026-06-29T12:01:05.405Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91c8caa73992901f320b69fd4ca00065","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T12:01:05.415Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-9163f405c42aa69851ae1644: done: 0.221414 sec","job_status":"done","queue_duration_s":0.002455,"scheduling_latency_s":0.002493,"redis_calls":14,"redis_duration_s":0.025618000000000002,"redis_read_bytes":901,"redis_write_bytes":1205,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.011825,"redis_feature_flag_read_bytes":896,"redis_feature_flag_write_bytes":464,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.011421,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002372,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056441,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.221414,"completed_at":"2026-06-29T12:01:05.639Z","load_balancing_strategy":"primary","db_duration_s":0.006285,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1686vm-test-run-nixbot-gitlab> gitlab # [  275.877135] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:05.661Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"c5444acc9c5fcb27a47876e2","created_at":"2026-06-29T12:01:05.464Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9e9b57fa6af5db964e0de7bf058f43ab","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.477Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-c5444acc9c5fcb27a47876e2: start","job_status":"start","queue_duration_s":0.184058,"scheduling_latency_s":0.18411}1687vm-test-run-nixbot-gitlab> gitlab # [  276.009540] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:05.873Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1688vm-test-run-nixbot-gitlab> gitlab # [  276.085358] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:05.948Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"c5444acc9c5fcb27a47876e2","created_at":"2026-06-29T12:01:05.464Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9e9b57fa6af5db964e0de7bf058f43ab","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.477Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-c5444acc9c5fcb27a47876e2: done: 0.287354 sec","job_status":"done","queue_duration_s":0.184058,"scheduling_latency_s":0.18411,"redis_calls":7,"redis_duration_s":0.018314999999999998,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008426,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.009889,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021356,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.287354,"completed_at":"2026-06-29T12:01:05.948Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1689vm-test-run-nixbot-gitlab> gitlab # [  276.163260] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:05.954Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"b4d93f72afff1b75fec80b91","created_at":"2026-06-29T12:01:05.490Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52cda7b88780229e4c395f9c9ae22f26","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.495Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-b4d93f72afff1b75fec80b91: start","job_status":"start","queue_duration_s":0.45919,"scheduling_latency_s":0.459241}1690vm-test-run-nixbot-gitlab> gitlab # [  276.180533] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:05.974Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1691vm-test-run-nixbot-gitlab> gitlab # [  276.184335] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:05.986Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"b4d93f72afff1b75fec80b91","created_at":"2026-06-29T12:01:05.490Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52cda7b88780229e4c395f9c9ae22f26","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.495Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-b4d93f72afff1b75fec80b91: done: 0.032715 sec","job_status":"done","queue_duration_s":0.45919,"scheduling_latency_s":0.459241,"redis_calls":7,"redis_duration_s":0.002692,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001925,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000767,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020425,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032715,"completed_at":"2026-06-29T12:01:05.986Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1692vm-test-run-nixbot-gitlab> gitlab # [  276.226302] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:05.991Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"dc22337c6095b5b15ddc82d8","created_at":"2026-06-29T12:01:05.536Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b0c7066fe7af1ba95d755cf6448997d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.545Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-dc22337c6095b5b15ddc82d8: start","job_status":"start","queue_duration_s":0.445681,"scheduling_latency_s":0.445715}1693vm-test-run-nixbot-gitlab> gitlab # [  276.240842] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:06.018Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"dc22337c6095b5b15ddc82d8","created_at":"2026-06-29T12:01:05.536Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b0c7066fe7af1ba95d755cf6448997d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.545Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-dc22337c6095b5b15ddc82d8: done: 0.027453 sec","job_status":"done","queue_duration_s":0.445681,"scheduling_latency_s":0.445715,"redis_calls":7,"redis_duration_s":0.002354,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001684,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00067,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027453,"completed_at":"2026-06-29T12:01:06.018Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1694vm-test-run-nixbot-gitlab> gitlab # [  276.282393] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:06.020Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"8c2be854b213b434843581cb","created_at":"2026-06-29T12:01:05.567Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e12b511a68a14c08745d43efefd65780","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.574Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-8c2be854b213b434843581cb: start","job_status":"start","queue_duration_s":0.445901,"scheduling_latency_s":0.44593}1695vm-test-run-nixbot-gitlab> gitlab # [  276.515468] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:06.379Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"8c2be854b213b434843581cb","created_at":"2026-06-29T12:01:05.567Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e12b511a68a14c08745d43efefd65780","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.574Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-8c2be854b213b434843581cb: done: 0.35847 sec","job_status":"done","queue_duration_s":0.445901,"scheduling_latency_s":0.44593,"redis_calls":9,"redis_duration_s":0.0027990000000000003,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001472,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001327,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":123,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":123,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.054,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.287813,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00080708,"exclusive_lock_hold_duration_s":0.33637906,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.35847,"completed_at":"2026-06-29T12:01:06.378Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.086597,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1696vm-test-run-nixbot-gitlab> gitlab # [  276.621251] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:06.380Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"83b12910b122b2000ff39fb8","created_at":"2026-06-29T12:01:05.604Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91739d177a433a64c606bd7e1b494028","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.620Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-83b12910b122b2000ff39fb8: start","job_status":"start","queue_duration_s":0.760128,"scheduling_latency_s":0.760149}1697vm-test-run-nixbot-gitlab> gitlab # [  276.641003] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:06.400Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"83b12910b122b2000ff39fb8","created_at":"2026-06-29T12:01:05.604Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91739d177a433a64c606bd7e1b494028","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.620Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-83b12910b122b2000ff39fb8: done: 0.019849 sec","job_status":"done","queue_duration_s":0.760128,"scheduling_latency_s":0.760149,"redis_calls":10,"redis_duration_s":0.001548,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000649,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0035907,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.019849,"completed_at":"2026-06-29T12:01:06.400Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001309,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1698vm-test-run-nixbot-gitlab> gitlab # [  276.691517] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:06.401Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"dc2b2370f505dd28bd3aa294","created_at":"2026-06-29T12:01:05.642Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2aeebb6b49c6553c6a11e41ee7b35034","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.650Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-dc2b2370f505dd28bd3aa294: start","job_status":"start","queue_duration_s":0.750887,"scheduling_latency_s":0.750906}1699vm-test-run-nixbot-gitlab> gitlab # [  276.708755] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:06.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"dc2b2370f505dd28bd3aa294","created_at":"2026-06-29T12:01:05.642Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2aeebb6b49c6553c6a11e41ee7b35034","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.650Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-dc2b2370f505dd28bd3aa294: done: 0.052634 sec","job_status":"done","queue_duration_s":0.750887,"scheduling_latency_s":0.750906,"redis_calls":9,"redis_duration_s":0.002421,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001459,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000962,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00048154,"exclusive_lock_hold_duration_s":0.03008002,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.052634,"completed_at":"2026-06-29T12:01:06.454Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009434,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1700vm-test-run-nixbot-gitlab> gitlab # [  276.754374] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:06.457Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"110f5890e94b7061433113d0","created_at":"2026-06-29T12:01:05.659Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"05c5e4df624107f14c175f64dc7916e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.666Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-110f5890e94b7061433113d0: start","job_status":"start","queue_duration_s":0.79023,"scheduling_latency_s":0.790905}1701vm-test-run-nixbot-gitlab> gitlab # [  276.769435] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:06.481Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"110f5890e94b7061433113d0","created_at":"2026-06-29T12:01:05.659Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"05c5e4df624107f14c175f64dc7916e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.666Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-110f5890e94b7061433113d0: done: 0.024749 sec","job_status":"done","queue_duration_s":0.79023,"scheduling_latency_s":0.790905,"redis_calls":7,"redis_duration_s":0.001786,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001278,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000508,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015204,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024749,"completed_at":"2026-06-29T12:01:06.481Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1702vm-test-run-nixbot-gitlab> gitlab # [  276.810865] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:06.484Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"7342093ff0fdffd012e259e1","created_at":"2026-06-29T12:01:05.676Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3cec4af15deb66cae076e29e05f673e3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.682Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-7342093ff0fdffd012e259e1: start","job_status":"start","queue_duration_s":0.801454,"scheduling_latency_s":0.801479}1703vm-test-run-nixbot-gitlab> gitlab # [  276.824846] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:06.524Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"7342093ff0fdffd012e259e1","created_at":"2026-06-29T12:01:05.676Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3cec4af15deb66cae076e29e05f673e3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.682Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-7342093ff0fdffd012e259e1: done: 0.040204 sec","job_status":"done","queue_duration_s":0.801454,"scheduling_latency_s":0.801479,"redis_calls":14,"redis_duration_s":0.00431,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000504,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002874,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000932,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040204,"completed_at":"2026-06-29T12:01:06.524Z","load_balancing_strategy":"primary","db_duration_s":0.000895,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1704vm-test-run-nixbot-gitlab> gitlab # [  276.866640] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:06.526Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"ac7f4baa9e502c63b38c1a39","created_at":"2026-06-29T12:01:05.693Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3208f6e5b530ae63142a517bb7dc0cb","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.697Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-ac7f4baa9e502c63b38c1a39: start","job_status":"start","queue_duration_s":0.828468,"scheduling_latency_s":0.828499}1705vm-test-run-nixbot-gitlab> gitlab # [  276.882258] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:06.548Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"ac7f4baa9e502c63b38c1a39","created_at":"2026-06-29T12:01:05.693Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3208f6e5b530ae63142a517bb7dc0cb","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.697Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-ac7f4baa9e502c63b38c1a39: done: 0.021701 sec","job_status":"done","queue_duration_s":0.828468,"scheduling_latency_s":0.828499,"redis_calls":7,"redis_duration_s":0.002497,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001961,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000536,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021701,"completed_at":"2026-06-29T12:01:06.547Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000792,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1706vm-test-run-nixbot-gitlab> gitlab # [  276.923374] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:06.549Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"268b8d82b505c457965c3eb9","created_at":"2026-06-29T12:01:05.706Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31ef3afc653f0780475e3e1b0a7d28b9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T12:01:05.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-268b8d82b505c457965c3eb9: start","job_status":"start","queue_duration_s":0.83939,"scheduling_latency_s":0.839413}1707vm-test-run-nixbot-gitlab> gitlab # [  276.937155] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:06.560Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1708vm-test-run-nixbot-gitlab> gitlab # [  276.940962] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:06.567Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"268b8d82b505c457965c3eb9","created_at":"2026-06-29T12:01:05.706Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31ef3afc653f0780475e3e1b0a7d28b9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T12:01:05.710Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-268b8d82b505c457965c3eb9: done: 0.017483 sec","job_status":"done","queue_duration_s":0.83939,"scheduling_latency_s":0.839413,"redis_calls":7,"redis_duration_s":0.001861,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001335,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000526,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011076,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017483,"completed_at":"2026-06-29T12:01:06.567Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1709vm-test-run-nixbot-gitlab> gitlab # [  276.981205] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:06.568Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"631aebe38f988dca39452bea","created_at":"2026-06-29T12:01:05.728Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.733Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-631aebe38f988dca39452bea: start","job_status":"start","queue_duration_s":0.835587,"scheduling_latency_s":0.835608}1710vm-test-run-nixbot-gitlab> gitlab # [  276.995147] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:06.664Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"631aebe38f988dca39452bea","created_at":"2026-06-29T12:01:05.728Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.733Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-631aebe38f988dca39452bea: done: 0.095312 sec","job_status":"done","queue_duration_s":0.835587,"scheduling_latency_s":0.835608,"redis_calls":60,"redis_duration_s":0.014575,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.000663,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.013136,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000776,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.081047,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.095312,"completed_at":"2026-06-29T12:01:06.664Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1711vm-test-run-nixbot-gitlab> gitlab # [  277.037037] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:06.665Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"cc1a4db5269f664736cd154c","created_at":"2026-06-29T12:01:05.738Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c2587a2111983ff1d52d8d4b44012a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.741Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-cc1a4db5269f664736cd154c: start","job_status":"start","queue_duration_s":0.92431,"scheduling_latency_s":0.924326}1712vm-test-run-nixbot-gitlab> gitlab # [  277.139505] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.003Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"cc1a4db5269f664736cd154c","created_at":"2026-06-29T12:01:05.738Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c2587a2111983ff1d52d8d4b44012a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.741Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-cc1a4db5269f664736cd154c: done: 0.337732 sec","job_status":"done","queue_duration_s":0.92431,"scheduling_latency_s":0.924326,"redis_calls":571,"redis_duration_s":0.066461,"redis_read_bytes":569,"redis_write_bytes":49487,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001088,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.065373,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.313422,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.337732,"completed_at":"2026-06-29T12:01:07.003Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1713vm-test-run-nixbot-gitlab> gitlab # [  277.180981] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.005Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"d05bc005d2f8dd736816f9f3","created_at":"2026-06-29T12:01:05.745Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c3b406523ee31020c78ac6ac40ecb5a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.750Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-d05bc005d2f8dd736816f9f3: start","job_status":"start","queue_duration_s":1.25451,"scheduling_latency_s":1.25453}1714vm-test-run-nixbot-gitlab> gitlab # [  277.196242] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.054Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"d05bc005d2f8dd736816f9f3","created_at":"2026-06-29T12:01:05.745Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c3b406523ee31020c78ac6ac40ecb5a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.750Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-d05bc005d2f8dd736816f9f3: done: 0.049669 sec","job_status":"done","queue_duration_s":1.25451,"scheduling_latency_s":1.25453,"redis_calls":9,"redis_duration_s":0.003784,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001365,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002419,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00182816,"exclusive_lock_hold_duration_s":0.02069618,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.049669,"completed_at":"2026-06-29T12:01:07.054Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012669,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1715vm-test-run-nixbot-gitlab> gitlab # [  277.240669] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.056Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"bc67ac5ff3c7696dab12e97f","created_at":"2026-06-29T12:01:05.762Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ed013ca21cf64616b97f15c96518734","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.765Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-bc67ac5ff3c7696dab12e97f: start","job_status":"start","queue_duration_s":1.291207,"scheduling_latency_s":1.291227}1716vm-test-run-nixbot-gitlab> gitlab # [  277.255204] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.064Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1717vm-test-run-nixbot-gitlab> gitlab # [  277.259069] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.070Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"bc67ac5ff3c7696dab12e97f","created_at":"2026-06-29T12:01:05.762Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ed013ca21cf64616b97f15c96518734","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.765Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-bc67ac5ff3c7696dab12e97f: done: 0.01379 sec","job_status":"done","queue_duration_s":1.291207,"scheduling_latency_s":1.291227,"redis_calls":7,"redis_duration_s":0.001635,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001181,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000454,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008069,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01379,"completed_at":"2026-06-29T12:01:07.070Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1718vm-test-run-nixbot-gitlab> gitlab # [  277.302388] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.071Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"972d384a005c09d8bd401121","created_at":"2026-06-29T12:01:05.769Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9cba0f7199fb74a46ddc99c91e514256","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.772Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-972d384a005c09d8bd401121: start","job_status":"start","queue_duration_s":1.299188,"scheduling_latency_s":1.299207}1719vm-test-run-nixbot-gitlab> gitlab # [  277.317596] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.080Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1720vm-test-run-nixbot-gitlab> gitlab # [  277.321486] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.086Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"972d384a005c09d8bd401121","created_at":"2026-06-29T12:01:05.769Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9cba0f7199fb74a46ddc99c91e514256","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.772Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-972d384a005c09d8bd401121: done: 0.014142 sec","job_status":"done","queue_duration_s":1.299188,"scheduling_latency_s":1.299207,"redis_calls":7,"redis_duration_s":0.0018310000000000002,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00128,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000551,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008274,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014142,"completed_at":"2026-06-29T12:01:07.086Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1721vm-test-run-nixbot-gitlab> gitlab # [  277.363331] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.087Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b578beed1ae9056762a8f6fa","created_at":"2026-06-29T12:01:05.797Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"455bb64f122acbd3000f8a8b28cf0ce5","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.801Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b578beed1ae9056762a8f6fa: start","job_status":"start","queue_duration_s":1.286631,"scheduling_latency_s":1.286648}1722vm-test-run-nixbot-gitlab> gitlab # [  277.378049] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.103Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b578beed1ae9056762a8f6fa","created_at":"2026-06-29T12:01:05.797Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"455bb64f122acbd3000f8a8b28cf0ce5","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.801Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b578beed1ae9056762a8f6fa: done: 0.015902 sec","job_status":"done","queue_duration_s":1.286631,"scheduling_latency_s":1.286648,"redis_calls":9,"redis_duration_s":0.0025320000000000004,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001502,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00103,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00053478,"exclusive_lock_hold_duration_s":0.0019775,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.015902,"completed_at":"2026-06-29T12:01:07.103Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000451,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1723vm-test-run-nixbot-gitlab> gitlab # [  277.420848] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.105Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"0d3a0e0eacaa2b4f4bafc46b","created_at":"2026-06-29T12:01:05.827Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b75201f16100885bee5fa2e8987f310","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.829Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-0d3a0e0eacaa2b4f4bafc46b: start","job_status":"start","queue_duration_s":1.27567,"scheduling_latency_s":1.275687}1724vm-test-run-nixbot-gitlab> gitlab # [  277.435439] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.125Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"0d3a0e0eacaa2b4f4bafc46b","created_at":"2026-06-29T12:01:05.827Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b75201f16100885bee5fa2e8987f310","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.829Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-0d3a0e0eacaa2b4f4bafc46b: done: 0.020278 sec","job_status":"done","queue_duration_s":1.27567,"scheduling_latency_s":1.275687,"redis_calls":7,"redis_duration_s":0.001716,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001258,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000458,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020278,"completed_at":"2026-06-29T12:01:07.125Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003226,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1725vm-test-run-nixbot-gitlab> gitlab # [  277.477020] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.126Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"2fd9443a32f958b11838e07f","created_at":"2026-06-29T12:01:05.839Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"99aac2a017c14e74d5d80723aa41158b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T12:01:05.842Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-2fd9443a32f958b11838e07f: start","job_status":"start","queue_duration_s":1.284692,"scheduling_latency_s":1.28471}1726vm-test-run-nixbot-gitlab> gitlab # [  277.491919] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.134Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1727vm-test-run-nixbot-gitlab> gitlab # [  277.495583] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.140Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"2fd9443a32f958b11838e07f","created_at":"2026-06-29T12:01:05.839Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"99aac2a017c14e74d5d80723aa41158b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T12:01:05.842Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-2fd9443a32f958b11838e07f: done: 0.013301 sec","job_status":"done","queue_duration_s":1.284692,"scheduling_latency_s":1.28471,"redis_calls":7,"redis_duration_s":0.00195,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001493,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000457,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013301,"completed_at":"2026-06-29T12:01:07.140Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1728vm-test-run-nixbot-gitlab> gitlab # [  277.536247] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.141Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"dca838ccbe2903fb0078931b","created_at":"2026-06-29T12:01:05.863Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"26f3474fcf5b40ebb9166530bcc72036","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.870Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-dca838ccbe2903fb0078931b: start","job_status":"start","queue_duration_s":1.271598,"scheduling_latency_s":1.271615}1729vm-test-run-nixbot-gitlab> gitlab # [  277.550246] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.157Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"dca838ccbe2903fb0078931b","created_at":"2026-06-29T12:01:05.863Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"26f3474fcf5b40ebb9166530bcc72036","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.870Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-dca838ccbe2903fb0078931b: done: 0.016197 sec","job_status":"done","queue_duration_s":1.271598,"scheduling_latency_s":1.271615,"redis_calls":7,"redis_duration_s":0.0023250000000000002,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001819,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000506,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.016197,"completed_at":"2026-06-29T12:01:07.157Z","load_balancing_strategy":"primary","db_duration_s":0.001387,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1730vm-test-run-nixbot-gitlab> gitlab # [  277.592441] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.159Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"4d75357863036e13dd562b82","created_at":"2026-06-29T12:01:05.921Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"46343e87fb555c331c8e0486805acf00","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.930Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-4d75357863036e13dd562b82: start","job_status":"start","queue_duration_s":1.229073,"scheduling_latency_s":1.229091}1731vm-test-run-nixbot-gitlab> gitlab # [  277.606420] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.175Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"4d75357863036e13dd562b82","created_at":"2026-06-29T12:01:05.921Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-23788dcd72154595","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"46343e87fb555c331c8e0486805acf00","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:05.930Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-4d75357863036e13dd562b82: done: 0.016236 sec","job_status":"done","queue_duration_s":1.229073,"scheduling_latency_s":1.229091,"redis_calls":9,"redis_duration_s":0.002314,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001216,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001098,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00055772,"exclusive_lock_hold_duration_s":0.0025913,"duration_s":0.016236,"completed_at":"2026-06-29T12:01:07.175Z","load_balancing_strategy":"primary","db_duration_s":0.000829,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1732vm-test-run-nixbot-gitlab> gitlab # [  277.648086] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.177Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"60c4b686d2901c3ed3d491b2","created_at":"2026-06-29T12:01:06.498Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-53e5fbd98e624f13","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3cec4af15deb66cae076e29e05f673e3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.515Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-60c4b686d2901c3ed3d491b2: start","job_status":"start","queue_duration_s":0.661939,"scheduling_latency_s":0.661956}1733vm-test-run-nixbot-gitlab> gitlab # [  277.660704] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.188Z","message":"No existing merge request to be cleaned up."}1734vm-test-run-nixbot-gitlab> gitlab # [  277.662397] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.196Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"60c4b686d2901c3ed3d491b2","created_at":"2026-06-29T12:01:06.498Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-53e5fbd98e624f13","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3cec4af15deb66cae076e29e05f673e3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.515Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-60c4b686d2901c3ed3d491b2: done: 0.019385 sec","job_status":"done","queue_duration_s":0.661939,"scheduling_latency_s":0.661956,"redis_calls":10,"redis_duration_s":0.0025069999999999997,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001452,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019385,"completed_at":"2026-06-29T12:01:07.196Z","load_balancing_strategy":"primary","db_duration_s":0.002542,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1735vm-test-run-nixbot-gitlab> gitlab # [  277.703229] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.198Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"174d2bc3eb4211663b6eb801","created_at":"2026-06-29T12:01:06.498Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-53e5fbd98e624f13","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3cec4af15deb66cae076e29e05f673e3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.515Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-174d2bc3eb4211663b6eb801: start","job_status":"start","queue_duration_s":0.682856,"scheduling_latency_s":0.682874}1736vm-test-run-nixbot-gitlab> gitlab # [  277.715782] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.209Z","message":"No existing merge request to be cleaned up."}1737vm-test-run-nixbot-gitlab> gitlab # [  277.717667] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.216Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"174d2bc3eb4211663b6eb801","created_at":"2026-06-29T12:01:06.498Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-53e5fbd98e624f13","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3cec4af15deb66cae076e29e05f673e3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.515Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-174d2bc3eb4211663b6eb801: done: 0.018865 sec","job_status":"done","queue_duration_s":0.682856,"scheduling_latency_s":0.682874,"redis_calls":10,"redis_duration_s":0.002458,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001362,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018865,"completed_at":"2026-06-29T12:01:07.216Z","load_balancing_strategy":"primary","db_duration_s":0.002415,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1738vm-test-run-nixbot-gitlab> gitlab # [  277.757088] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.218Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d12454bd17e6c0f30f8fa939","created_at":"2026-06-29T12:01:06.498Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-53e5fbd98e624f13","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3cec4af15deb66cae076e29e05f673e3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.515Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d12454bd17e6c0f30f8fa939: start","job_status":"start","queue_duration_s":0.703151,"scheduling_latency_s":0.703168}1739vm-test-run-nixbot-gitlab> gitlab # [  277.769828] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.228Z","message":"No existing merge request to be cleaned up."}1740vm-test-run-nixbot-gitlab> gitlab # [  277.771539] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.236Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d12454bd17e6c0f30f8fa939","created_at":"2026-06-29T12:01:06.498Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-53e5fbd98e624f13","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3cec4af15deb66cae076e29e05f673e3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.515Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d12454bd17e6c0f30f8fa939: done: 0.018481 sec","job_status":"done","queue_duration_s":0.703151,"scheduling_latency_s":0.703168,"redis_calls":10,"redis_duration_s":0.0024739999999999996,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00104,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001434,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011423,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018481,"completed_at":"2026-06-29T12:01:07.236Z","load_balancing_strategy":"primary","db_duration_s":0.002438,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1741vm-test-run-nixbot-gitlab> gitlab # [  277.811394] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.238Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"07ccc35ddf635c3ce61bf1ea","created_at":"2026-06-29T12:01:06.498Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-53e5fbd98e624f13","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3cec4af15deb66cae076e29e05f673e3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.515Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-07ccc35ddf635c3ce61bf1ea: start","job_status":"start","queue_duration_s":0.723138,"scheduling_latency_s":0.723155}1742vm-test-run-nixbot-gitlab> gitlab # [  277.824347] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.249Z","message":"No existing merge request to be cleaned up."}1743vm-test-run-nixbot-gitlab> gitlab # [  277.826050] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.257Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"07ccc35ddf635c3ce61bf1ea","created_at":"2026-06-29T12:01:06.498Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-53e5fbd98e624f13","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3cec4af15deb66cae076e29e05f673e3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.515Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-07ccc35ddf635c3ce61bf1ea: done: 0.019161 sec","job_status":"done","queue_duration_s":0.723138,"scheduling_latency_s":0.723155,"redis_calls":10,"redis_duration_s":0.002771,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001079,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001692,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019161,"completed_at":"2026-06-29T12:01:07.257Z","load_balancing_strategy":"primary","db_duration_s":0.002616,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1744vm-test-run-nixbot-gitlab> gitlab # [  277.865478] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.258Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9ef33d627e16c65985ea1d1","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9ef33d627e16c65985ea1d1: start","job_status":"start","queue_duration_s":0.601615,"scheduling_latency_s":0.60163}1745vm-test-run-nixbot-gitlab> gitlab # [  277.878252] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9ef33d627e16c65985ea1d1","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9ef33d627e16c65985ea1d1: done: 0.014978 sec","job_status":"done","queue_duration_s":0.601615,"scheduling_latency_s":0.60163,"redis_calls":13,"redis_duration_s":0.002501,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00159,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014978,"completed_at":"2026-06-29T12:01:07.273Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1746vm-test-run-nixbot-gitlab> gitlab # [  277.919261] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.275Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"044b3d20eaf2c9ef38a05e77","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-044b3d20eaf2c9ef38a05e77: start","job_status":"start","queue_duration_s":0.617936,"scheduling_latency_s":0.617952}1747vm-test-run-nixbot-gitlab> gitlab # [  277.932112] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.290Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"044b3d20eaf2c9ef38a05e77","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-044b3d20eaf2c9ef38a05e77: done: 0.014818 sec","job_status":"done","queue_duration_s":0.617936,"scheduling_latency_s":0.617952,"redis_calls":13,"redis_duration_s":0.002509,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000916,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001593,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014818,"completed_at":"2026-06-29T12:01:07.290Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1748vm-test-run-nixbot-gitlab> gitlab # [  277.971495] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.291Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b732f5a69c42baa89ffcd74","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b732f5a69c42baa89ffcd74: start","job_status":"start","queue_duration_s":0.634139,"scheduling_latency_s":0.634155}1749vm-test-run-nixbot-gitlab> gitlab # [  277.984232] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.306Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b732f5a69c42baa89ffcd74","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b732f5a69c42baa89ffcd74: done: 0.015209 sec","job_status":"done","queue_duration_s":0.634139,"scheduling_latency_s":0.634155,"redis_calls":13,"redis_duration_s":0.002863,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001918,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009422,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015209,"completed_at":"2026-06-29T12:01:07.306Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1750vm-test-run-nixbot-gitlab> gitlab # [  278.023539] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32c42f6cd2167504ed451519","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32c42f6cd2167504ed451519: start","job_status":"start","queue_duration_s":0.650958,"scheduling_latency_s":0.650974}1751vm-test-run-nixbot-gitlab> gitlab # [  278.036039] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.323Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32c42f6cd2167504ed451519","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32c42f6cd2167504ed451519: done: 0.014795 sec","job_status":"done","queue_duration_s":0.650958,"scheduling_latency_s":0.650974,"redis_calls":13,"redis_duration_s":0.002885,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001178,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001707,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009261,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014795,"completed_at":"2026-06-29T12:01:07.323Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1752vm-test-run-nixbot-gitlab> gitlab # [  278.075363] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.324Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a03b8dfe49ad9c3ef9a2caef","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a03b8dfe49ad9c3ef9a2caef: start","job_status":"start","queue_duration_s":0.666981,"scheduling_latency_s":0.666997}1753vm-test-run-nixbot-gitlab> gitlab # [  278.087992] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.346Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a03b8dfe49ad9c3ef9a2caef","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a03b8dfe49ad9c3ef9a2caef: done: 0.021942 sec","job_status":"done","queue_duration_s":0.666981,"scheduling_latency_s":0.666997,"redis_calls":13,"redis_duration_s":0.003894,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002074,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00182,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013725,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021942,"completed_at":"2026-06-29T12:01:07.346Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1754vm-test-run-nixbot-gitlab> gitlab # [  278.129076] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45d0338c5084dff7c16d001c","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45d0338c5084dff7c16d001c: start","job_status":"start","queue_duration_s":0.691926,"scheduling_latency_s":0.691967}1755vm-test-run-nixbot-gitlab> gitlab # [  278.166876] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45d0338c5084dff7c16d001c","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45d0338c5084dff7c16d001c: done: 0.035059 sec","job_status":"done","queue_duration_s":0.691926,"scheduling_latency_s":0.691967,"redis_calls":13,"redis_duration_s":0.003181,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001135,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002046,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024419,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035059,"completed_at":"2026-06-29T12:01:07.384Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1756vm-test-run-nixbot-gitlab> gitlab # [  278.238488] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a339a560b25e1de5290e876","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a339a560b25e1de5290e876: start","job_status":"start","queue_duration_s":0.72948,"scheduling_latency_s":0.729515}1757vm-test-run-nixbot-gitlab> gitlab # [  278.253140] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a339a560b25e1de5290e876","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a339a560b25e1de5290e876: done: 0.029493 sec","job_status":"done","queue_duration_s":0.72948,"scheduling_latency_s":0.729515,"redis_calls":13,"redis_duration_s":0.003331,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001131,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029493,"completed_at":"2026-06-29T12:01:07.416Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1758vm-test-run-nixbot-gitlab> gitlab # [  278.292901] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f27f2d9208b6bd41fe390027","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f27f2d9208b6bd41fe390027: start","job_status":"start","queue_duration_s":0.761202,"scheduling_latency_s":0.761234}1759vm-test-run-nixbot-gitlab> gitlab # [  278.305535] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.444Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f27f2d9208b6bd41fe390027","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f27f2d9208b6bd41fe390027: done: 0.025649 sec","job_status":"done","queue_duration_s":0.761202,"scheduling_latency_s":0.761234,"redis_calls":13,"redis_duration_s":0.002884,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001076,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001808,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025649,"completed_at":"2026-06-29T12:01:07.444Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1760vm-test-run-nixbot-gitlab> gitlab # [  278.346111] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22d7365893ca0314b73ba5d8","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22d7365893ca0314b73ba5d8: start","job_status":"start","queue_duration_s":0.788401,"scheduling_latency_s":0.788425}1761vm-test-run-nixbot-gitlab> gitlab # [  278.362411] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.466Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22d7365893ca0314b73ba5d8","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22d7365893ca0314b73ba5d8: done: 0.02054 sec","job_status":"done","queue_duration_s":0.788401,"scheduling_latency_s":0.788425,"redis_calls":13,"redis_duration_s":0.002152,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00078,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001372,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02054,"completed_at":"2026-06-29T12:01:07.466Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1762vm-test-run-nixbot-gitlab> gitlab # [  278.402015] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0230d0aea79f6576d445e644","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0230d0aea79f6576d445e644: start","job_status":"start","queue_duration_s":0.81047,"scheduling_latency_s":0.810489}1763vm-test-run-nixbot-gitlab> gitlab # [  278.414851] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0230d0aea79f6576d445e644","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0230d0aea79f6576d445e644: done: 0.016751 sec","job_status":"done","queue_duration_s":0.81047,"scheduling_latency_s":0.810489,"redis_calls":13,"redis_duration_s":0.001696,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000606,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016751,"completed_at":"2026-06-29T12:01:07.484Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1764vm-test-run-nixbot-gitlab> gitlab # [  278.454204] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4963a17f1dadf203c9084b2","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4963a17f1dadf203c9084b2: start","job_status":"start","queue_duration_s":0.828625,"scheduling_latency_s":0.828644}1765vm-test-run-nixbot-gitlab> gitlab # [  278.466610] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.500Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4963a17f1dadf203c9084b2","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4963a17f1dadf203c9084b2: done: 0.014545 sec","job_status":"done","queue_duration_s":0.828625,"scheduling_latency_s":0.828644,"redis_calls":13,"redis_duration_s":0.001434,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014545,"completed_at":"2026-06-29T12:01:07.500Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1766vm-test-run-nixbot-gitlab> gitlab # [  278.506477] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.501Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f8a05078c024a67082fdd46","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f8a05078c024a67082fdd46: start","job_status":"start","queue_duration_s":0.844295,"scheduling_latency_s":0.844313}1767vm-test-run-nixbot-gitlab> gitlab # [  278.519440] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.515Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f8a05078c024a67082fdd46","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f8a05078c024a67082fdd46: done: 0.014212 sec","job_status":"done","queue_duration_s":0.844295,"scheduling_latency_s":0.844313,"redis_calls":13,"redis_duration_s":0.001423,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009842,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014212,"completed_at":"2026-06-29T12:01:07.515Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1768vm-test-run-nixbot-gitlab> gitlab # [  278.560695] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.516Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24a89126ad0fb2dd4a6b35c8","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24a89126ad0fb2dd4a6b35c8: start","job_status":"start","queue_duration_s":0.859591,"scheduling_latency_s":0.859608}1769vm-test-run-nixbot-gitlab> gitlab # [  278.573359] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.547Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24a89126ad0fb2dd4a6b35c8","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24a89126ad0fb2dd4a6b35c8: done: 0.030305 sec","job_status":"done","queue_duration_s":0.859591,"scheduling_latency_s":0.859608,"redis_calls":13,"redis_duration_s":0.0015429999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02586,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030305,"completed_at":"2026-06-29T12:01:07.547Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1770vm-test-run-nixbot-gitlab> gitlab # [  278.613250] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.548Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88c902e83002f30e72ec5dd2","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88c902e83002f30e72ec5dd2: start","job_status":"start","queue_duration_s":0.890968,"scheduling_latency_s":0.890985}1771vm-test-run-nixbot-gitlab> gitlab # [  278.625640] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88c902e83002f30e72ec5dd2","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88c902e83002f30e72ec5dd2: done: 0.014128 sec","job_status":"done","queue_duration_s":0.890968,"scheduling_latency_s":0.890985,"redis_calls":13,"redis_duration_s":0.0014290000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009613,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014128,"completed_at":"2026-06-29T12:01:07.562Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1772vm-test-run-nixbot-gitlab> gitlab # [  278.666137] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.563Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec897595c17d1c1ca02d2c4c","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec897595c17d1c1ca02d2c4c: start","job_status":"start","queue_duration_s":0.906155,"scheduling_latency_s":0.906172}1773vm-test-run-nixbot-gitlab> gitlab # [  278.679054] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.577Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec897595c17d1c1ca02d2c4c","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec897595c17d1c1ca02d2c4c: done: 0.013635 sec","job_status":"done","queue_duration_s":0.906155,"scheduling_latency_s":0.906172,"redis_calls":13,"redis_duration_s":0.001401,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013635,"completed_at":"2026-06-29T12:01:07.577Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1774vm-test-run-nixbot-gitlab> gitlab # [  278.719234] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.578Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d7ddb8944df89c98c7abaa7","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d7ddb8944df89c98c7abaa7: start","job_status":"start","queue_duration_s":0.920821,"scheduling_latency_s":0.920838}1775vm-test-run-nixbot-gitlab> gitlab # [  278.731870] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d7ddb8944df89c98c7abaa7","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d7ddb8944df89c98c7abaa7: done: 0.014013 sec","job_status":"done","queue_duration_s":0.920821,"scheduling_latency_s":0.920838,"redis_calls":13,"redis_duration_s":0.0014789999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014013,"completed_at":"2026-06-29T12:01:07.592Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1776vm-test-run-nixbot-gitlab> gitlab # [  278.772910] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddf92515afd3abc8c9753566","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddf92515afd3abc8c9753566: start","job_status":"start","queue_duration_s":0.9359,"scheduling_latency_s":0.935918}1777vm-test-run-nixbot-gitlab> gitlab # [  278.785584] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.606Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddf92515afd3abc8c9753566","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddf92515afd3abc8c9753566: done: 0.013613 sec","job_status":"done","queue_duration_s":0.9359,"scheduling_latency_s":0.935918,"redis_calls":13,"redis_duration_s":0.0013809999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00087,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013613,"completed_at":"2026-06-29T12:01:07.606Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1778vm-test-run-nixbot-gitlab> gitlab # [  278.825550] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.607Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf0b6feccbdff65742131a25","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf0b6feccbdff65742131a25: start","job_status":"start","queue_duration_s":0.950553,"scheduling_latency_s":0.95057}1779vm-test-run-nixbot-gitlab> gitlab # [  278.838226] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.621Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf0b6feccbdff65742131a25","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf0b6feccbdff65742131a25: done: 0.013641 sec","job_status":"done","queue_duration_s":0.950553,"scheduling_latency_s":0.95057,"redis_calls":13,"redis_duration_s":0.001398,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00087,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009334,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013641,"completed_at":"2026-06-29T12:01:07.621Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1780vm-test-run-nixbot-gitlab> gitlab # [  278.877867] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.622Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4dd3740513a1474b89b8f51b","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4dd3740513a1474b89b8f51b: start","job_status":"start","queue_duration_s":0.965238,"scheduling_latency_s":0.965255}1781vm-test-run-nixbot-gitlab> gitlab # [  278.890611] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.636Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4dd3740513a1474b89b8f51b","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4dd3740513a1474b89b8f51b: done: 0.013743 sec","job_status":"done","queue_duration_s":0.965238,"scheduling_latency_s":0.965255,"redis_calls":13,"redis_duration_s":0.0014390000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000894,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013743,"completed_at":"2026-06-29T12:01:07.636Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1782vm-test-run-nixbot-gitlab> gitlab # [  278.930720] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"160a77edc9c2d64809ad6681","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-160a77edc9c2d64809ad6681: start","job_status":"start","queue_duration_s":0.980031,"scheduling_latency_s":0.980047}1783vm-test-run-nixbot-gitlab> gitlab # [  278.944330] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"160a77edc9c2d64809ad6681","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-160a77edc9c2d64809ad6681: done: 0.013624 sec","job_status":"done","queue_duration_s":0.980031,"scheduling_latency_s":0.980047,"redis_calls":13,"redis_duration_s":0.001386,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000879,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013624,"completed_at":"2026-06-29T12:01:07.650Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1784vm-test-run-nixbot-gitlab> gitlab # [  278.984265] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.652Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b97810692c402134e21fee65","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b97810692c402134e21fee65: start","job_status":"start","queue_duration_s":0.994933,"scheduling_latency_s":0.994951}1785vm-test-run-nixbot-gitlab> gitlab # [  278.996863] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.666Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b97810692c402134e21fee65","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b97810692c402134e21fee65: done: 0.013781 sec","job_status":"done","queue_duration_s":0.994933,"scheduling_latency_s":0.994951,"redis_calls":13,"redis_duration_s":0.0014069999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013781,"completed_at":"2026-06-29T12:01:07.665Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1786vm-test-run-nixbot-gitlab> gitlab # [  279.036978] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e37b82c6d1207414b3faf439","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e37b82c6d1207414b3faf439: start","job_status":"start","queue_duration_s":1.009784,"scheduling_latency_s":1.009801}1787vm-test-run-nixbot-gitlab> gitlab # [  279.049365] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.681Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e37b82c6d1207414b3faf439","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e37b82c6d1207414b3faf439: done: 0.014019 sec","job_status":"done","queue_duration_s":1.009784,"scheduling_latency_s":1.009801,"redis_calls":13,"redis_duration_s":0.0014780000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000941,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009661,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014019,"completed_at":"2026-06-29T12:01:07.681Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1788vm-test-run-nixbot-gitlab> gitlab # [  279.089109] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.682Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a56d3f825f66b4217616528","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a56d3f825f66b4217616528: start","job_status":"start","queue_duration_s":1.024813,"scheduling_latency_s":1.024831}1789vm-test-run-nixbot-gitlab> gitlab # [  279.101559] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a56d3f825f66b4217616528","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a56d3f825f66b4217616528: done: 0.013651 sec","job_status":"done","queue_duration_s":1.024813,"scheduling_latency_s":1.024831,"redis_calls":13,"redis_duration_s":0.001414,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000911,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009318,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013651,"completed_at":"2026-06-29T12:01:07.695Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1790vm-test-run-nixbot-gitlab> gitlab # [  279.140211] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.696Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73ec85c2484b3b9f92c45845","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73ec85c2484b3b9f92c45845: start","job_status":"start","queue_duration_s":1.03952,"scheduling_latency_s":1.039537}1791vm-test-run-nixbot-gitlab> gitlab # [  279.153537] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.710Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73ec85c2484b3b9f92c45845","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73ec85c2484b3b9f92c45845: done: 0.013591 sec","job_status":"done","queue_duration_s":1.03952,"scheduling_latency_s":1.039537,"redis_calls":13,"redis_duration_s":0.001388,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009357,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013591,"completed_at":"2026-06-29T12:01:07.710Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1792vm-test-run-nixbot-gitlab> gitlab # [  279.192454] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7b6d0a4ec405be57bb5e85c","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7b6d0a4ec405be57bb5e85c: start","job_status":"start","queue_duration_s":1.054158,"scheduling_latency_s":1.054175}1793vm-test-run-nixbot-gitlab> gitlab # [  279.204680] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.725Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7b6d0a4ec405be57bb5e85c","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7b6d0a4ec405be57bb5e85c: done: 0.013863 sec","job_status":"done","queue_duration_s":1.054158,"scheduling_latency_s":1.054175,"redis_calls":13,"redis_duration_s":0.001415,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000897,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009489,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013863,"completed_at":"2026-06-29T12:01:07.725Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1794vm-test-run-nixbot-gitlab> gitlab # [  279.243593] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.726Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51841f53bb347c9a4e4bda4d","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51841f53bb347c9a4e4bda4d: start","job_status":"start","queue_duration_s":1.069115,"scheduling_latency_s":1.069133}1795vm-test-run-nixbot-gitlab> gitlab # [  279.256540] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.740Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51841f53bb347c9a4e4bda4d","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51841f53bb347c9a4e4bda4d: done: 0.01382 sec","job_status":"done","queue_duration_s":1.069115,"scheduling_latency_s":1.069133,"redis_calls":13,"redis_duration_s":0.001438,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01382,"completed_at":"2026-06-29T12:01:07.740Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1796vm-test-run-nixbot-gitlab> gitlab # [  279.295692] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04e3c6b82f50a21c99edd805","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04e3c6b82f50a21c99edd805: start","job_status":"start","queue_duration_s":1.083986,"scheduling_latency_s":1.084004}1797vm-test-run-nixbot-gitlab> gitlab # [  279.308305] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04e3c6b82f50a21c99edd805","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04e3c6b82f50a21c99edd805: done: 0.014111 sec","job_status":"done","queue_duration_s":1.083986,"scheduling_latency_s":1.084004,"redis_calls":13,"redis_duration_s":0.001689,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014111,"completed_at":"2026-06-29T12:01:07.755Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1798vm-test-run-nixbot-gitlab> gitlab # [  279.347567] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.756Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"604b8fae6bd5d1fae308cec1","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-604b8fae6bd5d1fae308cec1: start","job_status":"start","queue_duration_s":1.099168,"scheduling_latency_s":1.099186}1799vm-test-run-nixbot-gitlab> gitlab # [  279.361009] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.770Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"604b8fae6bd5d1fae308cec1","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-604b8fae6bd5d1fae308cec1: done: 0.013528 sec","job_status":"done","queue_duration_s":1.099168,"scheduling_latency_s":1.099186,"redis_calls":13,"redis_duration_s":0.001388,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013528,"completed_at":"2026-06-29T12:01:07.769Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1800vm-test-run-nixbot-gitlab> gitlab # [  279.400679] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.771Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bb7e7ad0dfdb304a3279d81","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bb7e7ad0dfdb304a3279d81: start","job_status":"start","queue_duration_s":1.113709,"scheduling_latency_s":1.113739}1801vm-test-run-nixbot-gitlab> gitlab # [  279.413352] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bb7e7ad0dfdb304a3279d81","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bb7e7ad0dfdb304a3279d81: done: 0.013681 sec","job_status":"done","queue_duration_s":1.113709,"scheduling_latency_s":1.113739,"redis_calls":13,"redis_duration_s":0.001392,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000888,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013681,"completed_at":"2026-06-29T12:01:07.784Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1802vm-test-run-nixbot-gitlab> gitlab # [  279.452458] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7de0abe400c434381a084bc","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7de0abe400c434381a084bc: start","job_status":"start","queue_duration_s":1.128419,"scheduling_latency_s":1.128436}1803vm-test-run-nixbot-gitlab> gitlab # [  279.464886] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.800Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7de0abe400c434381a084bc","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7de0abe400c434381a084bc: done: 0.014306 sec","job_status":"done","queue_duration_s":1.128419,"scheduling_latency_s":1.128436,"redis_calls":13,"redis_duration_s":0.001523,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000592,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000931,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014306,"completed_at":"2026-06-29T12:01:07.800Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1804vm-test-run-nixbot-gitlab> gitlab # [  279.504548] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.801Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d2764ef65b7c33026f6a36f","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d2764ef65b7c33026f6a36f: start","job_status":"start","queue_duration_s":1.14385,"scheduling_latency_s":1.143868}1805vm-test-run-nixbot-gitlab> gitlab # [  279.516809] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.814Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d2764ef65b7c33026f6a36f","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d2764ef65b7c33026f6a36f: done: 0.013549 sec","job_status":"done","queue_duration_s":1.14385,"scheduling_latency_s":1.143868,"redis_calls":13,"redis_duration_s":0.001388,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000875,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009294,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013549,"completed_at":"2026-06-29T12:01:07.814Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1806vm-test-run-nixbot-gitlab> gitlab # [  279.556879] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.815Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"408c0d923830f08a476c1e63","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-408c0d923830f08a476c1e63: start","job_status":"start","queue_duration_s":1.158453,"scheduling_latency_s":1.158471}1807vm-test-run-nixbot-gitlab> gitlab # [  279.569440] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.829Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"408c0d923830f08a476c1e63","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-408c0d923830f08a476c1e63: done: 0.013971 sec","job_status":"done","queue_duration_s":1.158453,"scheduling_latency_s":1.158471,"redis_calls":13,"redis_duration_s":0.001438,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013971,"completed_at":"2026-06-29T12:01:07.829Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1808vm-test-run-nixbot-gitlab> gitlab # [  279.608587] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.830Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95cfc560968e65043273f479","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95cfc560968e65043273f479: start","job_status":"start","queue_duration_s":1.173492,"scheduling_latency_s":1.17351}1809vm-test-run-nixbot-gitlab> gitlab # [  279.620830] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95cfc560968e65043273f479","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95cfc560968e65043273f479: done: 0.014041 sec","job_status":"done","queue_duration_s":1.173492,"scheduling_latency_s":1.17351,"redis_calls":13,"redis_duration_s":0.001431,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000908,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009541,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014041,"completed_at":"2026-06-29T12:01:07.844Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1810vm-test-run-nixbot-gitlab> gitlab # [  279.659951] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.845Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18fdb7269c53740e559062c6","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18fdb7269c53740e559062c6: start","job_status":"start","queue_duration_s":1.188571,"scheduling_latency_s":1.188588}1811vm-test-run-nixbot-gitlab> gitlab # [  279.673057] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.859Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18fdb7269c53740e559062c6","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18fdb7269c53740e559062c6: done: 0.014074 sec","job_status":"done","queue_duration_s":1.188571,"scheduling_latency_s":1.188588,"redis_calls":13,"redis_duration_s":0.0016359999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009574,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014074,"completed_at":"2026-06-29T12:01:07.859Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1812vm-test-run-nixbot-gitlab> gitlab # [  279.712568] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2c53c3553d683fd57f7e722","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2c53c3553d683fd57f7e722: start","job_status":"start","queue_duration_s":1.203695,"scheduling_latency_s":1.203712}1813vm-test-run-nixbot-gitlab> gitlab # [  279.725102] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.874Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2c53c3553d683fd57f7e722","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2c53c3553d683fd57f7e722: done: 0.013679 sec","job_status":"done","queue_duration_s":1.203695,"scheduling_latency_s":1.203712,"redis_calls":13,"redis_duration_s":0.001448,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009426,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013679,"completed_at":"2026-06-29T12:01:07.874Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1814vm-test-run-nixbot-gitlab> gitlab # [  279.765307] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.875Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e89618e5701e195d00782dba","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e89618e5701e195d00782dba: start","job_status":"start","queue_duration_s":1.218459,"scheduling_latency_s":1.218475}1815vm-test-run-nixbot-gitlab> gitlab # [  279.777767] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e89618e5701e195d00782dba","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e89618e5701e195d00782dba: done: 0.013783 sec","job_status":"done","queue_duration_s":1.218459,"scheduling_latency_s":1.218475,"redis_calls":13,"redis_duration_s":0.001436,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000889,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009426,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013783,"completed_at":"2026-06-29T12:01:07.889Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1816vm-test-run-nixbot-gitlab> gitlab # [  279.816888] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69b12edcc0e13f023f8e524b","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69b12edcc0e13f023f8e524b: start","job_status":"start","queue_duration_s":1.23327,"scheduling_latency_s":1.233287}1817vm-test-run-nixbot-gitlab> gitlab # [  279.829298] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69b12edcc0e13f023f8e524b","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69b12edcc0e13f023f8e524b: done: 0.01371 sec","job_status":"done","queue_duration_s":1.23327,"scheduling_latency_s":1.233287,"redis_calls":13,"redis_duration_s":0.0014290000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000897,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009344,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01371,"completed_at":"2026-06-29T12:01:07.904Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1818vm-test-run-nixbot-gitlab> gitlab # [  279.868560] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30d167bc58712385bf577f08","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30d167bc58712385bf577f08: start","job_status":"start","queue_duration_s":1.248057,"scheduling_latency_s":1.248074}1819vm-test-run-nixbot-gitlab> gitlab # [  279.880984] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30d167bc58712385bf577f08","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30d167bc58712385bf577f08: done: 0.013869 sec","job_status":"done","queue_duration_s":1.248057,"scheduling_latency_s":1.248074,"redis_calls":13,"redis_duration_s":0.001434,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009567,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013869,"completed_at":"2026-06-29T12:01:07.919Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1820vm-test-run-nixbot-gitlab> gitlab # [  279.920741] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b09b1f9f0da93ab55d0fceea","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b09b1f9f0da93ab55d0fceea: start","job_status":"start","queue_duration_s":1.262974,"scheduling_latency_s":1.262991}1821vm-test-run-nixbot-gitlab> gitlab # [  279.933434] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b09b1f9f0da93ab55d0fceea","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b09b1f9f0da93ab55d0fceea: done: 0.013627 sec","job_status":"done","queue_duration_s":1.262974,"scheduling_latency_s":1.262991,"redis_calls":13,"redis_duration_s":0.001413,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009336,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013627,"completed_at":"2026-06-29T12:01:07.933Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1822vm-test-run-nixbot-gitlab> gitlab # [  279.973851] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15691955ff9297ce9e134910","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15691955ff9297ce9e134910: start","job_status":"start","queue_duration_s":1.277629,"scheduling_latency_s":1.277646}1823vm-test-run-nixbot-gitlab> gitlab # [  279.986445] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.948Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15691955ff9297ce9e134910","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15691955ff9297ce9e134910: done: 0.013699 sec","job_status":"done","queue_duration_s":1.277629,"scheduling_latency_s":1.277646,"redis_calls":13,"redis_duration_s":0.001398,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000893,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013699,"completed_at":"2026-06-29T12:01:07.948Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1824vm-test-run-nixbot-gitlab> gitlab # [  280.025766] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d665db22f9116d5ee2cd7663","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d665db22f9116d5ee2cd7663: start","job_status":"start","queue_duration_s":1.292364,"scheduling_latency_s":1.292381}1825vm-test-run-nixbot-gitlab> gitlab # [  280.038030] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d665db22f9116d5ee2cd7663","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d665db22f9116d5ee2cd7663: done: 0.014382 sec","job_status":"done","queue_duration_s":1.292364,"scheduling_latency_s":1.292381,"redis_calls":13,"redis_duration_s":0.001696,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014382,"completed_at":"2026-06-29T12:01:07.964Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1826vm-test-run-nixbot-gitlab> gitlab # [  280.077260] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.965Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8af0c5f185cf66fb94a3e4ab","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8af0c5f185cf66fb94a3e4ab: start","job_status":"start","queue_duration_s":1.307795,"scheduling_latency_s":1.307812}1827vm-test-run-nixbot-gitlab> gitlab # [  280.089709] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8af0c5f185cf66fb94a3e4ab","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8af0c5f185cf66fb94a3e4ab: done: 0.013549 sec","job_status":"done","queue_duration_s":1.307795,"scheduling_latency_s":1.307812,"redis_calls":13,"redis_duration_s":0.0014030000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000879,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009348,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013549,"completed_at":"2026-06-29T12:01:07.978Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1828vm-test-run-nixbot-gitlab> gitlab # [  280.128766] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb74c067ead323ecbdfc27a9","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb74c067ead323ecbdfc27a9: start","job_status":"start","queue_duration_s":1.322376,"scheduling_latency_s":1.322391}1829vm-test-run-nixbot-gitlab> gitlab # [  280.141274] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb74c067ead323ecbdfc27a9","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb74c067ead323ecbdfc27a9: done: 0.01364 sec","job_status":"done","queue_duration_s":1.322376,"scheduling_latency_s":1.322391,"redis_calls":13,"redis_duration_s":0.001423,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009336,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01364,"completed_at":"2026-06-29T12:01:07.993Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1830vm-test-run-nixbot-gitlab> gitlab # [  280.181455] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:07.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34c0262b662baa863ad5d06d","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34c0262b662baa863ad5d06d: start","job_status":"start","queue_duration_s":1.337061,"scheduling_latency_s":1.337079}1831vm-test-run-nixbot-gitlab> gitlab # [  280.193812] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:08.008Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34c0262b662baa863ad5d06d","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34c0262b662baa863ad5d06d: done: 0.0138 sec","job_status":"done","queue_duration_s":1.337061,"scheduling_latency_s":1.337079,"redis_calls":13,"redis_duration_s":0.001414,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000903,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0138,"completed_at":"2026-06-29T12:01:08.008Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1832vm-test-run-nixbot-gitlab> gitlab # [  280.233482] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:08.009Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c09292e34fff2fd16202fa8","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c09292e34fff2fd16202fa8: start","job_status":"start","queue_duration_s":1.351917,"scheduling_latency_s":1.351934}1833vm-test-run-nixbot-gitlab> gitlab # [  280.246147] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:08.023Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c09292e34fff2fd16202fa8","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c09292e34fff2fd16202fa8: done: 0.014571 sec","job_status":"done","queue_duration_s":1.351917,"scheduling_latency_s":1.351934,"redis_calls":13,"redis_duration_s":0.002194,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00169,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014571,"completed_at":"2026-06-29T12:01:08.023Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1834vm-test-run-nixbot-gitlab> gitlab # [  280.285340] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:08.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e626b674fdda0b47e28e90d5","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e626b674fdda0b47e28e90d5: start","job_status":"start","queue_duration_s":1.367594,"scheduling_latency_s":1.367612}1835vm-test-run-nixbot-gitlab> gitlab # [  280.297809] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:08.038Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e626b674fdda0b47e28e90d5","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e626b674fdda0b47e28e90d5: done: 0.013917 sec","job_status":"done","queue_duration_s":1.367594,"scheduling_latency_s":1.367612,"redis_calls":13,"redis_duration_s":0.0014329999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013917,"completed_at":"2026-06-29T12:01:08.038Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1836vm-test-run-nixbot-gitlab> gitlab # [  280.337519] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:08.039Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad9c6aff1d38c41a24437aaa","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad9c6aff1d38c41a24437aaa: start","job_status":"start","queue_duration_s":1.382552,"scheduling_latency_s":1.382568}1837vm-test-run-nixbot-gitlab> gitlab # [  280.350108] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:08.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad9c6aff1d38c41a24437aaa","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad9c6aff1d38c41a24437aaa: done: 0.013772 sec","job_status":"done","queue_duration_s":1.382552,"scheduling_latency_s":1.382568,"redis_calls":13,"redis_duration_s":0.00148,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000593,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000887,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009441,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013772,"completed_at":"2026-06-29T12:01:08.053Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1838vm-test-run-nixbot-gitlab> gitlab # [  280.390077] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:08.054Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ed8991b09e29fe06def0899","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ed8991b09e29fe06def0899: start","job_status":"start","queue_duration_s":1.397569,"scheduling_latency_s":1.397586}1839vm-test-run-nixbot-gitlab> gitlab # [  280.402769] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:08.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ed8991b09e29fe06def0899","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ed8991b09e29fe06def0899: done: 0.013894 sec","job_status":"done","queue_duration_s":1.397569,"scheduling_latency_s":1.397586,"redis_calls":13,"redis_duration_s":0.0014680000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013894,"completed_at":"2026-06-29T12:01:08.068Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1840vm-test-run-nixbot-gitlab> gitlab # [  280.442471] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:08.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8f985982bb95386e95a0a56","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8f985982bb95386e95a0a56: start","job_status":"start","queue_duration_s":1.412534,"scheduling_latency_s":1.412553}1841vm-test-run-nixbot-gitlab> gitlab # [  280.454845] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:08.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8f985982bb95386e95a0a56","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8f985982bb95386e95a0a56: done: 0.013716 sec","job_status":"done","queue_duration_s":1.412534,"scheduling_latency_s":1.412553,"redis_calls":13,"redis_duration_s":0.001375,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000871,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013716,"completed_at":"2026-06-29T12:01:08.083Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1842vm-test-run-nixbot-gitlab> gitlab # [  280.494438] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:08.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"052b502bbad13da04dc714c9","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-052b502bbad13da04dc714c9: start","job_status":"start","queue_duration_s":1.427301,"scheduling_latency_s":1.427319}1843vm-test-run-nixbot-gitlab> gitlab # [  280.506928] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:08.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"052b502bbad13da04dc714c9","created_at":"2026-06-29T12:01:06.580Z","trace_propagation_headers":{"sentry-trace":"d855527d8f6442a39a5a082d90afdb60-754f76d82ccd4fa0","baggage":"sentry-trace_id=d855527d8f6442a39a5a082d90afdb60,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f78c863b66c78499d0f445b7070ad27","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:06.657Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-052b502bbad13da04dc714c9: done: 0.013592 sec","job_status":"done","queue_duration_s":1.427301,"scheduling_latency_s":1.427319,"redis_calls":13,"redis_duration_s":0.001385,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000884,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009344,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013592,"completed_at":"2026-06-29T12:01:08.098Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1844vm-test-run-nixbot-gitlab> gitlab # [  298.340487] runuser[3283]: pam_unix(runuser:session): session closed for user gitlab1845vm-test-run-nixbot-gitlab> gitlab # [  298.345333] setup-gitlab-start[3634]: +++ date -d '+7 days' +%F1846vm-test-run-nixbot-gitlab> gitlab # [  298.348761] setup-gitlab-start[3633]: ++ jq -r .token1847vm-test-run-nixbot-gitlab> gitlab # [  298.352128] setup-gitlab-start[3632]: ++ curl -fs -X POST -H 'Authorization: Bearer 7a18b8207f9a42599941d936e50d9d6ea8ff4a1607b9899a71bc42ece84d43e7' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-06"}' http://gitlab/api/v4/users/1/personal_access_tokens1848vm-test-run-nixbot-gitlab> gitlab # [  298.678090] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:28.540Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"c8f0940198eb7a3e271c738e","created_at":"2026-06-29T12:01:28.469Z","trace_propagation_headers":{"sentry-trace":"b513e5d1e06f4a0c93182ad5c5ea3ece-4e1327d5784d43ec","baggage":"sentry-trace_id=b513e5d1e06f4a0c93182ad5c5ea3ece,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M30N1K7JXAJQT39A6XXQQ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:9f88f85afada9c142331ff7af8ab78c1f672ec8ef9d4ee60fdbf95bef450e365","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:28.533Z","job_size_bytes":451,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-c8f0940198eb7a3e271c738e: start","job_status":"start","queue_duration_s":0.006743,"scheduling_latency_s":0.006809}1849vm-test-run-nixbot-gitlab> gitlab # [  298.727237] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:28 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 3311850vm-test-run-nixbot-gitlab> gitlab # [  298.732574] setup-gitlab-start[3232]: + TOKEN=glpat-7o5ltjBaxvNtJ9BLyDewEG86MQp1OjEH.01.0w1i1nmfl1851vm-test-run-nixbot-gitlab> gitlab # [  298.735221] setup-gitlab-start[3232]: + test -n glpat-7o5ltjBaxvNtJ9BLyDewEG86MQp1OjEH.01.0w1i1nmfl1852vm-test-run-nixbot-gitlab> gitlab # [  298.737852] setup-gitlab-start[3232]: + test glpat-7o5ltjBaxvNtJ9BLyDewEG86MQp1OjEH.01.0w1i1nmfl '!=' null1853vm-test-run-nixbot-gitlab> gitlab # [  298.741411] setup-gitlab-start[3232]: + mkdir -p /var/lib/secrets1854vm-test-run-nixbot-gitlab> gitlab # [  298.743080] setup-gitlab-start[3232]: + echo glpat-7o5ltjBaxvNtJ9BLyDewEG86MQp1OjEH.01.0w1i1nmfl1855vm-test-run-nixbot-gitlab> gitlab # [  298.745547] setup-gitlab-start[3232]: + chmod 644 /var/lib/secrets/gitlab-token1856vm-test-run-nixbot-gitlab> gitlab # [  298.747583] setup-gitlab-start[3232]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-7o5ltjBaxvNtJ9BLyDewEG86MQp1OjEH.01.0w1i1nmfl' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1857vm-test-run-nixbot-gitlab> gitlab # [  298.899582] sidekiq-cluster[2415]: {"severity":"WARN","time":"2026-06-29T12:01:28.761Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"c8f0940198eb7a3e271c738e","created_at":"2026-06-29T12:01:28.469Z","trace_propagation_headers":{"sentry-trace":"b513e5d1e06f4a0c93182ad5c5ea3ece-4e1327d5784d43ec","baggage":"sentry-trace_id=b513e5d1e06f4a0c93182ad5c5ea3ece,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M30N1K7JXAJQT39A6XXQQ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:9f88f85afada9c142331ff7af8ab78c1f672ec8ef9d4ee60fdbf95bef450e365","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:28.533Z","job_size_bytes":451,"pid":2415,"sidekiq_tid":"lyu7","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-c8f0940198eb7a3e271c738e: fail: 0.219958 sec","job_status":"fail","queue_duration_s":0.006743,"scheduling_latency_s":0.006809,"redis_calls":11,"redis_duration_s":0.005783,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001381,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003329,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001073,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.164789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00107892,"duration_s":0.219958,"completed_at":"2026-06-29T12:01:28.759Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/w9mp1gyfs2qdv12r0ji70idzndmxwz2s-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/p8ylyr0sj9sxcknqcpja13nypqixk2dp-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/dndr7b4a17339dsm67vzb6nv4p2kp82q-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/zx6ndpscxs64hnidz69ab430ccms6rak-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.028212,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1858vm-test-run-nixbot-gitlab> gitlab # [  299.980209] gitaly[1370]: time="2026-06-29T12:01:29.839Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M310NGCEJ00WVX24ZVBA3 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T12:01:39.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:29.839" pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1859vm-test-run-nixbot-gitlab> gitlab # [  299.988569] gitaly[1370]: time="2026-06-29T12:01:29.839Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M310NGCEJ00WVX24ZVBA3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T12:01:39.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:29.839" grpc.time_ms=0.693 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1860vm-test-run-nixbot-gitlab> gitlab # [  300.240001] gitaly[1370]: time="2026-06-29T12:01:30.103Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M310NGCEJ00WVX24ZVBA3 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T12:01:40.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:30.103" pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1861vm-test-run-nixbot-gitlab> gitlab # [  300.306170] gitaly[1370]: time="2026-06-29T12:01:30.104Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M310NGCEJ00WVX24ZVBA3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T12:01:40.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:30.103" grpc.time_ms=0.525 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1862vm-test-run-nixbot-gitlab> gitlab # [  300.329777] gitaly[1370]: time="2026-06-29T12:01:30.149Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M310NGCEJ00WVX24ZVBA3 diskcache=97f1f037-a031-4a59-8f83-f684b28764d2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T12:01:40.015" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:30.116" pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1863vm-test-run-nixbot-gitlab> gitlab # [  300.347115] gitaly[1370]: time="2026-06-29T12:01:30.149Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=299 command.oublock=32 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M310NGCEJ00WVX24ZVBA3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T12:01:40.015" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:30.116" grpc.time_ms=33.215 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1864vm-test-run-nixbot-gitlab> gitlab # [  300.369597] gitaly[1370]: time="2026-06-29T12:01:30.170Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M310NGCEJ00WVX24ZVBA3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T12:01:40.068" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:30.169" pid=1370 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1865vm-test-run-nixbot-gitlab> gitlab # [  300.385133] gitaly[1370]: time="2026-06-29T12:01:30.170Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M310NGCEJ00WVX24ZVBA3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T12:01:40.068" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:30.169" grpc.time_ms=1.325 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1866vm-test-run-nixbot-gitlab> gitlab # [  300.400247] gitaly[1370]: time="2026-06-29T12:01:30.194Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M310NGCEJ00WVX24ZVBA3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T12:01:40.076" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:30.177" grpc.time_ms=16.822 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1867vm-test-run-nixbot-gitlab> gitlab # [  300.418193] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:30.267Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"54e976e72fab50602b5c28e8","created_at":"2026-06-29T12:01:30.210Z","trace_propagation_headers":{"sentry-trace":"c178296d4e0b4632923fbb8a03188c78-da8dae926d6f4209","baggage":"sentry-trace_id=c178296d4e0b4632923fbb8a03188c78,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M310NGCEJ00WVX24ZVBA3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:30.263Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-54e976e72fab50602b5c28e8: start","job_status":"start","queue_duration_s":0.00378,"scheduling_latency_s":0.003837}1868vm-test-run-nixbot-gitlab> gitlab # [  300.459113] gitaly[1370]: time="2026-06-29T12:01:30.323Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M310NGCEJ00WVX24ZVBA3 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T12:01:40.222" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:30.322" pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1869vm-test-run-nixbot-gitlab> gitlab # [  300.491879] gitaly[1370]: time="2026-06-29T12:01:30.323Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M310NGCEJ00WVX24ZVBA3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T12:01:40.222" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:30.322" grpc.time_ms=0.499 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1870vm-test-run-nixbot-gitlab> gitlab # [  300.523657] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:30.337Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"54e976e72fab50602b5c28e8","created_at":"2026-06-29T12:01:30.210Z","trace_propagation_headers":{"sentry-trace":"c178296d4e0b4632923fbb8a03188c78-da8dae926d6f4209","baggage":"sentry-trace_id=c178296d4e0b4632923fbb8a03188c78,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M310NGCEJ00WVX24ZVBA3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:30.263Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-54e976e72fab50602b5c28e8: done: 0.070219 sec","job_status":"done","queue_duration_s":0.00378,"scheduling_latency_s":0.003837,"redis_calls":13,"redis_duration_s":0.004251,"redis_read_bytes":138,"redis_write_bytes":1593,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000964,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":428,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001766,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001521,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.03647752,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.070219,"completed_at":"2026-06-29T12:01:30.337Z","load_balancing_strategy":"primary","db_duration_s":0.007419,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1871vm-test-run-nixbot-gitlab> gitlab # [  300.600438] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:30.339Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"d8262b4c7a29e65f5b4c2401","created_at":"2026-06-29T12:01:30.297Z","trace_propagation_headers":{"sentry-trace":"c178296d4e0b4632923fbb8a03188c78-da8dae926d6f4209","baggage":"sentry-trace_id=c178296d4e0b4632923fbb8a03188c78,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M310NGCEJ00WVX24ZVBA3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:30.311Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-d8262b4c7a29e65f5b4c2401: start","job_status":"start","queue_duration_s":0.028727,"scheduling_latency_s":0.028752}1872vm-test-run-nixbot-gitlab> gitlab # [  300.623794] gitaly[1370]: time="2026-06-29T12:01:30.362Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M310NGCEJ00WVX24ZVBA3 diskcache=4400e5cb-b351-48d9-9869-8b0a9f0fb7c9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T12:01:40.235" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:30.335" pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1873vm-test-run-nixbot-gitlab> gitlab # [  300.638194] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:30.477Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"d8262b4c7a29e65f5b4c2401","created_at":"2026-06-29T12:01:30.297Z","trace_propagation_headers":{"sentry-trace":"c178296d4e0b4632923fbb8a03188c78-da8dae926d6f4209","baggage":"sentry-trace_id=c178296d4e0b4632923fbb8a03188c78,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M310NGCEJ00WVX24ZVBA3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:30.311Z","job_size_bytes":2,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-d8262b4c7a29e65f5b4c2401: done: 0.137682 sec","job_status":"done","queue_duration_s":0.028727,"scheduling_latency_s":0.028752,"redis_calls":14,"redis_duration_s":0.003457,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000698,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001667,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001092,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.114082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.10755614,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.137682,"completed_at":"2026-06-29T12:01:30.477Z","load_balancing_strategy":"primary","db_duration_s":0.016006,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1874vm-test-run-nixbot-gitlab> gitlab # [  300.688688] gitaly[1370]: time="2026-06-29T12:01:30.363Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=299 command.oublock=32 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M310NGCEJ00WVX24ZVBA3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T12:01:40.235" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:30.335" grpc.time_ms=27.003 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1875vm-test-run-nixbot-gitlab> gitlab # [  300.702798] gitaly[1370]: time="2026-06-29T12:01:30.372Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M310NGCEJ00WVX24ZVBA3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T12:01:40.270" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:30.371" pid=1370 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1876vm-test-run-nixbot-gitlab> gitlab # [  300.713570] gitaly[1370]: time="2026-06-29T12:01:30.372Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M310NGCEJ00WVX24ZVBA3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T12:01:40.270" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:30.371" grpc.time_ms=0.846 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1877vm-test-run-nixbot-gitlab> gitlab # [  300.724651] gitaly[1370]: time="2026-06-29T12:01:30.385Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M310NGCEJ00WVX24ZVBA3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T12:01:40.276" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:30.377" grpc.time_ms=8.042 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1878vm-test-run-nixbot-gitlab> gitlab # [  300.738553] gitaly[1370]: time="2026-06-29T12:01:30.423Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M310NGCEJ00WVX24ZVBA3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T12:01:40.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:30.423" grpc.time_ms=0.224 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1879vm-test-run-nixbot-gitlab> gitlab # [  300.749509] gitaly[1370]: time="2026-06-29T12:01:30.434Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M310NGCEJ00WVX24ZVBA3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T12:01:40.013" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:30.433" grpc.time_ms=0.402 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1880vm-test-run-nixbot-gitlab> gitlab # [  301.037409] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:30.901Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"c29f1755430734fcc7a23787","created_at":"2026-06-29T12:01:30.892Z","trace_propagation_headers":{"sentry-trace":"c178296d4e0b4632923fbb8a03188c78-da8dae926d6f4209","baggage":"sentry-trace_id=c178296d4e0b4632923fbb8a03188c78,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M310NGCEJ00WVX24ZVBA3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:30.899Z","job_size_bytes":5,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-c29f1755430734fcc7a23787: start","job_status":"start","queue_duration_s":0.00136,"scheduling_latency_s":0.001401}1881vm-test-run-nixbot-gitlab> gitlab # [  301.229311] gitaly[1370]: time="2026-06-29T12:01:31.092Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M310NGCEJ00WVX24ZVBA3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T12:01:41.004" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:31.074" grpc.time_ms=17.861 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1882vm-test-run-nixbot-gitlab> gitlab # [  301.251892] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:31.104Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"c29f1755430734fcc7a23787","created_at":"2026-06-29T12:01:30.892Z","trace_propagation_headers":{"sentry-trace":"c178296d4e0b4632923fbb8a03188c78-da8dae926d6f4209","baggage":"sentry-trace_id=c178296d4e0b4632923fbb8a03188c78,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M310NGCEJ00WVX24ZVBA3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:30.899Z","job_size_bytes":5,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-c29f1755430734fcc7a23787: done: 0.203212 sec","job_status":"done","queue_duration_s":0.00136,"scheduling_latency_s":0.001401,"redis_calls":12,"redis_duration_s":0.004911,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000869,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003257,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000785,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.140132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00068234,"duration_s":0.203212,"completed_at":"2026-06-29T12:01:31.104Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024084,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1883vm-test-run-nixbot-gitlab> gitlab # [  301.304866] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:31.106Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"2def51f114deb555fdcadd67","created_at":"2026-06-29T12:01:30.951Z","trace_propagation_headers":{"sentry-trace":"c178296d4e0b4632923fbb8a03188c78-da8dae926d6f4209","baggage":"sentry-trace_id=c178296d4e0b4632923fbb8a03188c78,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M310NGCEJ00WVX24ZVBA3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:30.962Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-2def51f114deb555fdcadd67: start","job_status":"start","queue_duration_s":0.143609,"scheduling_latency_s":0.143637}1884vm-test-run-nixbot-gitlab> gitlab # [  301.482634] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:31.345Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"2def51f114deb555fdcadd67","created_at":"2026-06-29T12:01:30.951Z","trace_propagation_headers":{"sentry-trace":"c178296d4e0b4632923fbb8a03188c78-da8dae926d6f4209","baggage":"sentry-trace_id=c178296d4e0b4632923fbb8a03188c78,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M310NGCEJ00WVX24ZVBA3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:30.962Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-2def51f114deb555fdcadd67: done: 0.238707 sec","job_status":"done","queue_duration_s":0.143609,"scheduling_latency_s":0.143637,"redis_calls":11,"redis_duration_s":0.0027400000000000002,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000564,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001618,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000558,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.189,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.189,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.163057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00056202,"duration_s":0.238707,"completed_at":"2026-06-29T12:01:31.345Z","load_balancing_strategy":"primary","db_duration_s":0.078059,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1885vm-test-run-nixbot-gitlab> gitlab # [  301.586089] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:31.347Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"8397e12aa3841fdc7765c74a","created_at":"2026-06-29T12:01:30.971Z","trace_propagation_headers":{"sentry-trace":"c178296d4e0b4632923fbb8a03188c78-da8dae926d6f4209","baggage":"sentry-trace_id=c178296d4e0b4632923fbb8a03188c78,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M310NGCEJ00WVX24ZVBA3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:30.981Z","job_size_bytes":89,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-8397e12aa3841fdc7765c74a: start","job_status":"start","queue_duration_s":0.365439,"scheduling_latency_s":0.365464}1886vm-test-run-nixbot-gitlab> gitlab # [  301.636443] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:31.430Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"8397e12aa3841fdc7765c74a","created_at":"2026-06-29T12:01:30.971Z","trace_propagation_headers":{"sentry-trace":"c178296d4e0b4632923fbb8a03188c78-da8dae926d6f4209","baggage":"sentry-trace_id=c178296d4e0b4632923fbb8a03188c78,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M310NGCEJ00WVX24ZVBA3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:30.981Z","job_size_bytes":89,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-8397e12aa3841fdc7765c74a: done: 0.083694 sec","job_status":"done","queue_duration_s":0.365439,"scheduling_latency_s":0.365464,"redis_calls":11,"redis_duration_s":0.003132,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000859,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001725,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000548,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00045926,"duration_s":0.083694,"completed_at":"2026-06-29T12:01:31.430Z","load_balancing_strategy":"primary","db_duration_s":0.035116,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1887vm-test-run-nixbot-gitlab> gitlab # [  303.491028] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:33 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 47561888vm-test-run-nixbot-gitlab> gitlab # [  303.495208] setup-gitlab-start[3232]: + mkdir -p /tmp/test-flake1889vm-test-run-nixbot-gitlab> gitlab # [  303.499597] setup-gitlab-start[3232]: + cd /tmp/test-flake1890vm-test-run-nixbot-gitlab> gitlab # [  303.501986] setup-gitlab-start[3232]: + git init -b master1891vm-test-run-nixbot-gitlab> gitlab # [  303.533867] setup-gitlab-start[3689]: Initialized empty Git repository in /tmp/test-flake/.git/1892vm-test-run-nixbot-gitlab> gitlab # [  303.537609] setup-gitlab-start[3232]: + git config user.name test1893vm-test-run-nixbot-gitlab> gitlab # [  303.540876] setup-gitlab-start[3232]: + git config user.email test@example.com1894vm-test-run-nixbot-gitlab> gitlab # [  303.547684] setup-gitlab-start[3232]: + cat1895vm-test-run-nixbot-gitlab> gitlab # [  303.553428] setup-gitlab-start[3232]: + git add flake.nix1896vm-test-run-nixbot-gitlab> gitlab # [  303.561883] setup-gitlab-start[3232]: + git commit -m 'initial commit'1897vm-test-run-nixbot-gitlab> gitlab # [  303.582972] setup-gitlab-start[3694]: [master (root-commit) 55d99a0] initial commit1898vm-test-run-nixbot-gitlab> gitlab # [  303.586311] setup-gitlab-start[3694]:  1 file changed, 10 insertions(+)1899vm-test-run-nixbot-gitlab> gitlab # [  303.589097] setup-gitlab-start[3694]:  create mode 100644 flake.nix1900vm-test-run-nixbot-gitlab> gitlab # [  303.591750] setup-gitlab-start[3232]: + git remote add origin http://oauth2:glpat-7o5ltjBaxvNtJ9BLyDewEG86MQp1OjEH.01.0w1i1nmfl@gitlab/root/test-flake.git1901vm-test-run-nixbot-gitlab> gitlab # [  303.597434] setup-gitlab-start[3232]: + git push -u origin master1902vm-test-run-nixbot-gitlab> gitlab # [  303.728779] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:33 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 821903vm-test-run-nixbot-gitlab> gitlab # [  303.925099] gitaly[1370]: time="2026-06-29T12:01:33.788Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M35WT45DE5MXWXS8MENJY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:33.770" grpc.time_ms=17.735 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1904vm-test-run-nixbot-gitlab> gitlab # [  303.963037] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:33 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 1951905vm-test-run-nixbot-gitlab> gitlab # [  304.460647] gitaly[1370]: time="2026-06-29T12:01:34.324Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T12:01:44.004" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:34.315" grpc.time_ms=9.14 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1906vm-test-run-nixbot-gitlab> gitlab # [  304.530560] gitaly[1370]: time="2026-06-29T12:01:34.394Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-29T12:02:04.077" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-29T12:01:34.377" grpc.time_ms=16.91 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1907vm-test-run-nixbot-gitlab> gitlab # [  304.588384] gitaly[1370]: time="2026-06-29T12:01:34.452Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KW9M363AQK5G59D1J857N27S duration_ms=443ns method=POST pid=1370 status=200 url="http://unix/api/v4/internal/allowed"1908vm-test-run-nixbot-gitlab> gitlab # [  304.651285] gitaly[1370]: time="2026-06-29T12:01:34.515Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KW9M363AQK5G59D1J857N27S duration_ms=61ns method=POST pid=1370 status=200 url="http://unix/api/v4/internal/pre_receive"1909vm-test-run-nixbot-gitlab> gitlab # [  304.664525] gitaly[1370]: time="2026-06-29T12:01:34.516Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5318 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:34.007" grpc.time_ms=508.869 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1910vm-test-run-nixbot-gitlab> gitlab # [  304.748178] gitaly[1370]: time="2026-06-29T12:01:34.612Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4788 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:34.610" grpc.time_ms=1.151 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1911vm-test-run-nixbot-gitlab> gitlab # [  304.799914] gitaly[1370]: time="2026-06-29T12:01:34.664Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:34.663" grpc.time_ms=0.816 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1912vm-test-run-nixbot-gitlab> gitlab # [  304.842026] gitaly[1370]: time="2026-06-29T12:01:34.706Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4789 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:34.705" grpc.time_ms=0.543 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1913vm-test-run-nixbot-gitlab> gitlab # [  304.960475] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:34.822Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 55d99a0d7974c5e56383b279aac3e27df907757a refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"4bb410faa49bb8aa29f9b92a","created_at":"2026-06-29T12:01:34.812Z","trace_propagation_headers":{"sentry-trace":"aa766160c03649388cd2b6e91224c6d7-58237045ef3448e5","baggage":"sentry-trace_id=aa766160c03649388cd2b6e91224c6d7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M363AQK5G59D1J857N27S","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:34.820Z","job_size_bytes":185,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-4bb410faa49bb8aa29f9b92a: start","job_status":"start","queue_duration_s":0.002266,"scheduling_latency_s":0.00235}1914vm-test-run-nixbot-gitlab> gitlab # [  305.009632] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:34.873Z","message":"changes: \"0000000000000000000000000000000000000000 55d99a0d7974c5e56383b279aac3e27df907757a refs/heads/master\\n\""}1915vm-test-run-nixbot-gitlab> gitlab # [  305.021499] gitaly[1370]: time="2026-06-29T12:01:34.884Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=6 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T12:01:44.004" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:34.865" grpc.time_ms=19.291 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1916vm-test-run-nixbot-gitlab> gitlab # [  305.053824] gitaly[1370]: time="2026-06-29T12:01:34.914Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T12:01:44.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:34.897" grpc.time_ms=16.315 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1917vm-test-run-nixbot-gitlab> gitlab # [  305.075908] gitaly[1370]: time="2026-06-29T12:01:34.928Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1433 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=14 command.system_time_ms=5 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-29T12:01:44.039" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:34.900" grpc.time_ms=27.909 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1918vm-test-run-nixbot-gitlab> gitlab # [  305.096375] gitaly[1370]: time="2026-06-29T12:01:34.949Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KW9M363AQK5G59D1J857N27S duration_ms=188ns method=POST pid=1370 status=200 url="http://unix/api/v4/internal/post_receive"1919vm-test-run-nixbot-gitlab> gitlab # [  305.100485] gitaly[1370]: time="2026-06-29T12:01:34.950Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4812 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:34.759" grpc.time_ms=190.596 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1920vm-test-run-nixbot-gitlab> gitlab # [  305.115385] gitaly[1370]: time="2026-06-29T12:01:34.962Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T12:01:44.061" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:34.951" grpc.time_ms=10.688 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1921vm-test-run-nixbot-gitlab> gitlab # [  305.131414] gitaly[1370]: time="2026-06-29T12:01:34.967Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S diskcache=5074976f-d3d4-4310-afeb-02eb14e6c72e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:33.902" pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1922vm-test-run-nixbot-gitlab> gitlab # [  305.143225] systemd[1]: Finished setup-gitlab.service.1923vm-test-run-nixbot-gitlab> gitlab # [  305.144396] gitlab-workhorse[1286]: time="2026-06-29T12:01:34Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521924vm-test-run-nixbot-gitlab> gitlab # [  305.146352] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:34 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 11651925vm-test-run-nixbot-gitlab> gitlab # [  305.148521] setup-gitlab-start[3698]: To http://gitlab/root/test-flake.git1926vm-test-run-nixbot-gitlab> gitlab # [  305.149457] setup-gitlab-start[3698]:  * [new branch]      master -> master1927vm-test-run-nixbot-gitlab> gitlab # [  305.150390] setup-gitlab-start[3698]: branch 'master' set up to track 'origin/master'.1928vm-test-run-nixbot-gitlab> gitlab # [  305.151475] gitaly[1370]: time="2026-06-29T12:01:34.967Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=323 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13331 command.oublock=32 command.real_time_ms=1058 command.spawn_token_fork_ms=2 command.system_time_ms=195 command.user_time_ms=128 component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:33.902" grpc.time_ms=1065.507 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1929vm-test-run-nixbot-gitlab> gitlab # [  305.165346] systemd[1]: Starting nixbot CI service...1930vm-test-run-nixbot-gitlab> gitlab # [  305.166214] gitaly[1370]: time="2026-06-29T12:01:35.013Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=5 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-29T12:01:45.001" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:35.001" grpc.time_ms=11.433 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1931vm-test-run-nixbot-gitlab> gitlab # [  305.197287] gitaly[1370]: time="2026-06-29T12:01:35.061Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3938 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:35.061" grpc.time_ms=0.335 pid=1370 span.kind=server system=grpc1932vm-test-run-nixbot-gitlab> gitlab # [  305.233359] gitaly[1370]: time="2026-06-29T12:01:35.097Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3940 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:35.097" grpc.time_ms=0.462 pid=1370 span.kind=server system=grpc1933vm-test-run-nixbot-gitlab> gitlab # [  305.243916] gitaly[1370]: time="2026-06-29T12:01:35.107Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S diskcache=6fd254f4-643c-4566-974a-eb2ee2997798 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T12:01:45.026" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:35.027" pid=1370 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1934vm-test-run-nixbot-gitlab> gitlab # [  305.254553] gitaly[1370]: time="2026-06-29T12:01:35.108Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=77 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4996 command.oublock=8 command.real_time_ms=76 command.spawn_token_fork_ms=0 command.system_time_ms=42 command.user_time_ms=35 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T12:01:45.026" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:35.027" grpc.time_ms=81.059 pid=1370 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1935vm-test-run-nixbot-gitlab> gitlab # [  305.269345] gitaly[1370]: time="2026-06-29T12:01:35.127Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T12:01:45.126" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:35.127" grpc.time_ms=0.144 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1936vm-test-run-nixbot-gitlab> gitlab # [  305.283415] gitaly[1370]: time="2026-06-29T12:01:35.147Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=861 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-29T12:01:45.132" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:35.133" grpc.time_ms=13.784 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1937vm-test-run-nixbot-gitlab> gitlab # [  305.408628] gitaly[1370]: time="2026-06-29T12:01:35.272Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=313 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-06-29T12:02:05.060" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:35.261" grpc.time_ms=11.327 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1938vm-test-run-nixbot-gitlab> gitlab # [  305.452754] gitaly[1370]: time="2026-06-29T12:01:35.316Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T12:01:45.005" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:35.316" grpc.time_ms=0.552 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1939vm-test-run-nixbot-gitlab> gitlab # [  305.486975] gitaly[1370]: time="2026-06-29T12:01:35.328Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=317 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-29T12:01:45.009" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-06-29T12:01:35.319" grpc.time_ms=8.251 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1940vm-test-run-nixbot-gitlab> gitlab # [  305.529045] gitaly[1370]: time="2026-06-29T12:01:35.367Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T12:01:45.042" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:35.353" grpc.time_ms=14.494 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1941vm-test-run-nixbot-gitlab> gitlab # [  305.565992] gitaly[1370]: time="2026-06-29T12:01:35.388Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T12:01:45.065" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:35.375" grpc.time_ms=12.562 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1942vm-test-run-nixbot-gitlab> gitlab # [  305.597614] gitaly[1370]: time="2026-06-29T12:01:35.425Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=883 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=3 command.system_time_ms=17 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T12:02:05.202" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:35.403" grpc.time_ms=21.935 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1943vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit setup-gitlab.service, in 306.67 seconds)1944vm-test-run-nixbot-gitlab> gitlab # [  306.112817] gitaly[1370]: time="2026-06-29T12:01:35.975Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-29T12:01:45.082" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:35.972" grpc.time_ms=2.767 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1945vm-test-run-nixbot-gitlab> gitlab: waiting for unit nixbot.service1946vm-test-run-nixbot-gitlab> gitlab # [  306.175824] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:36.038Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 55d99a0d7974c5e56383b279aac3e27df907757a refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"4bb410faa49bb8aa29f9b92a","created_at":"2026-06-29T12:01:34.812Z","trace_propagation_headers":{"sentry-trace":"aa766160c03649388cd2b6e91224c6d7-58237045ef3448e5","baggage":"sentry-trace_id=aa766160c03649388cd2b6e91224c6d7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M363AQK5G59D1J857N27S","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:34.820Z","job_size_bytes":185,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-4bb410faa49bb8aa29f9b92a: done: 1.216106 sec","job_status":"done","queue_duration_s":0.002266,"scheduling_latency_s":0.00235,"gitaly_calls":13,"gitaly_duration_s":0.251382,"redis_calls":196,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.069204,"redis_read_bytes":1445,"redis_write_bytes":19016,"redis_cache_calls":14,"redis_cache_duration_s":0.00673,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":28,"redis_feature_flag_duration_s":0.015711,"redis_feature_flag_read_bytes":1044,"redis_feature_flag_write_bytes":3487,"redis_queues_calls":8,"redis_queues_duration_s":0.001632,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4932,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.005232,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":91,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.020712,"redis_repository_cache_read_bytes":233,"redis_repository_cache_write_bytes":6319,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.019187,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.09,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.278,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.313,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.09,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.817171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00150182,"exclusive_lock_hold_duration_s":0.0464213,"duration_s":1.216106,"completed_at":"2026-06-29T12:01:36.038Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.132443,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1947vm-test-run-nixbot-gitlab> gitlab # [  306.311655] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:36.040Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"86039530b1f30bd003a043b8","created_at":"2026-06-29T12:01:34.967Z","trace_propagation_headers":{"sentry-trace":"aa766160c03649388cd2b6e91224c6d7-5655134909e94716","baggage":"sentry-trace_id=aa766160c03649388cd2b6e91224c6d7,sentry-release=Unknown"},"correlation_id":"01KW9M363AQK5G59D1J857N27S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:34.979Z","job_size_bytes":46,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-86039530b1f30bd003a043b8: start","job_status":"start","queue_duration_s":1.060978,"scheduling_latency_s":1.061002}1948vm-test-run-nixbot-gitlab> gitlab # [  306.337651] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:36.162Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"86039530b1f30bd003a043b8","created_at":"2026-06-29T12:01:34.967Z","trace_propagation_headers":{"sentry-trace":"aa766160c03649388cd2b6e91224c6d7-5655134909e94716","baggage":"sentry-trace_id=aa766160c03649388cd2b6e91224c6d7,sentry-release=Unknown"},"correlation_id":"01KW9M363AQK5G59D1J857N27S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:34.979Z","job_size_bytes":46,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-86039530b1f30bd003a043b8: done: 0.122006 sec","job_status":"done","queue_duration_s":1.060978,"scheduling_latency_s":1.061002,"gitaly_calls":2,"gitaly_duration_s":0.026499,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.010772,"redis_read_bytes":68,"redis_write_bytes":4887,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.00117,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000422,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00241,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.005116,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":752,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001654,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.065254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035686,"duration_s":0.122006,"completed_at":"2026-06-29T12:01:36.162Z","load_balancing_strategy":"primary","db_duration_s":0.021697,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1949vm-test-run-nixbot-gitlab> gitlab # [  306.401554] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:36.163Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"f2124b79c6242dc94b0caf0e","created_at":"2026-06-29T12:01:34.990Z","trace_propagation_headers":{"sentry-trace":"aa766160c03649388cd2b6e91224c6d7-5655134909e94716","baggage":"sentry-trace_id=aa766160c03649388cd2b6e91224c6d7,sentry-release=Unknown"},"correlation_id":"01KW9M363AQK5G59D1J857N27S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:34.996Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-f2124b79c6242dc94b0caf0e: start","job_status":"start","queue_duration_s":1.166879,"scheduling_latency_s":1.166901}1950vm-test-run-nixbot-gitlab> gitlab # [  306.421004] gitaly[1370]: time="2026-06-29T12:01:36.102Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-06-29T18:01:36.098" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:36.099" grpc.time_ms=3.095 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1951vm-test-run-nixbot-gitlab> gitlab # [  306.433047] nixbot[3805]: {"ts": "2026-06-29T12:01:36.142Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1952vm-test-run-nixbot-gitlab> gitlab # [  306.435377] nixbot[3805]: {"ts": "2026-06-29T12:01:36.174Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1953vm-test-run-nixbot-gitlab> gitlab # [  306.437540] nixbot[3805]: {"ts": "2026-06-29T12:01:36.214Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1954vm-test-run-nixbot-gitlab> gitlab # [  306.439773] nixbot[3805]: {"ts": "2026-06-29T12:01:36.219Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1955vm-test-run-nixbot-gitlab> gitlab # [  306.442049] nixbot[3805]: {"ts": "2026-06-29T12:01:36.229Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1956vm-test-run-nixbot-gitlab> gitlab # [  306.444065] nixbot[3805]: {"ts": "2026-06-29T12:01:36.240Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1957vm-test-run-nixbot-gitlab> gitlab # [  306.446099] nixbot[3805]: {"ts": "2026-06-29T12:01:36.245Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1958vm-test-run-nixbot-gitlab> gitlab # [  306.448168] nixbot[3805]: {"ts": "2026-06-29T12:01:36.247Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1959vm-test-run-nixbot-gitlab> gitlab # [  306.450288] nixbot[3805]: {"ts": "2026-06-29T12:01:36.256Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1960vm-test-run-nixbot-gitlab> gitlab # [  306.452368] nixbot[3805]: {"ts": "2026-06-29T12:01:36.258Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1961vm-test-run-nixbot-gitlab> gitlab # [  306.454426] nixbot[3805]: {"ts": "2026-06-29T12:01:36.265Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1962vm-test-run-nixbot-gitlab> gitlab # [  306.456411] nixbot[3805]: {"ts": "2026-06-29T12:01:36.273Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1963vm-test-run-nixbot-gitlab> gitlab # [  306.458453] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:36.271Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"f2124b79c6242dc94b0caf0e","created_at":"2026-06-29T12:01:34.990Z","trace_propagation_headers":{"sentry-trace":"aa766160c03649388cd2b6e91224c6d7-5655134909e94716","baggage":"sentry-trace_id=aa766160c03649388cd2b6e91224c6d7,sentry-release=Unknown"},"correlation_id":"01KW9M363AQK5G59D1J857N27S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:34.996Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-f2124b79c6242dc94b0caf0e: done: 0.107813 sec","job_status":"done","queue_duration_s":1.166879,"scheduling_latency_s":1.166901,"gitaly_calls":1,"gitaly_duration_s":0.028376,"redis_calls":15,"redis_duration_s":0.007194,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000491,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004841,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000662,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.0012,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.015,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.026,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0720943,"duration_s":0.107813,"completed_at":"2026-06-29T12:01:36.271Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026131,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1964vm-test-run-nixbot-gitlab> gitlab # [  306.509174] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:36.272Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"8c7f0c2796d8f2e84bd68af4","created_at":"2026-06-29T12:01:35.121Z","trace_propagation_headers":{"sentry-trace":"aa766160c03649388cd2b6e91224c6d7-5655134909e94716","baggage":"sentry-trace_id=aa766160c03649388cd2b6e91224c6d7,sentry-release=Unknown"},"correlation_id":"01KW9M363AQK5G59D1J857N27S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:35.123Z","job_size_bytes":23,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8c7f0c2796d8f2e84bd68af4: start","job_status":"start","queue_duration_s":1.149008,"scheduling_latency_s":1.149028}1965vm-test-run-nixbot-gitlab> gitlab # [  306.526010] gitaly[1370]: time="2026-06-29T12:01:36.132Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=591 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-29T12:02:06.018" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:36.118" grpc.time_ms=13.638 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1966vm-test-run-nixbot-gitlab> gitlab # [  306.539551] gitaly[1370]: time="2026-06-29T12:01:36.220Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S diskcache=433f53d2-b3c8-4d35-80d3-34ab56effe80 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-29T18:01:36.194" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:36.195" pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1967vm-test-run-nixbot-gitlab> gitlab # [  306.550412] nixbot[3805]: {"ts": "2026-06-29T12:01:36.277Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1968vm-test-run-nixbot-gitlab> gitlab # [  306.554207] nixbot[3805]: {"ts": "2026-06-29T12:01:36.279Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1969vm-test-run-nixbot-gitlab> gitlab # [  306.558970] nixbot[3805]: {"ts": "2026-06-29T12:01:36.284Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1970vm-test-run-nixbot-gitlab> gitlab # [  306.561234] nixbot[3805]: {"ts": "2026-06-29T12:01:36.286Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1971vm-test-run-nixbot-gitlab> gitlab # [  306.563303] nixbot[3805]: {"ts": "2026-06-29T12:01:36.288Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1972vm-test-run-nixbot-gitlab> gitlab # [  306.565801] nixbot[3805]: {"ts": "2026-06-29T12:01:36.289Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1973vm-test-run-nixbot-gitlab> gitlab # [  306.568183] nixbot[3805]: {"ts": "2026-06-29T12:01:36.295Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1974vm-test-run-nixbot-gitlab> gitlab # [  306.570647] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:36.341Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"8c7f0c2796d8f2e84bd68af4","created_at":"2026-06-29T12:01:35.121Z","trace_propagation_headers":{"sentry-trace":"aa766160c03649388cd2b6e91224c6d7-5655134909e94716","baggage":"sentry-trace_id=aa766160c03649388cd2b6e91224c6d7,sentry-release=Unknown"},"correlation_id":"01KW9M363AQK5G59D1J857N27S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:35.123Z","job_size_bytes":23,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8c7f0c2796d8f2e84bd68af4: done: 0.069031 sec","job_status":"done","queue_duration_s":1.149008,"scheduling_latency_s":1.149028,"gitaly_calls":1,"gitaly_duration_s":0.021404,"redis_calls":16,"redis_duration_s":0.006952,"redis_read_bytes":52,"redis_write_bytes":2046,"redis_queues_calls":1,"redis_queues_duration_s":0.000267,"redis_queues_read_bytes":1,"redis_queues_write_bytes":977,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002494,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.003436,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000755,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.069031,"completed_at":"2026-06-29T12:01:36.341Z","load_balancing_strategy":"primary","db_duration_s":0.002312,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1975vm-test-run-nixbot-gitlab> gitlab # [  306.619003] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:36.344Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"388ce0a5176053dfd857196b","created_at":"2026-06-29T12:01:35.875Z","trace_propagation_headers":{"sentry-trace":"aa766160c03649388cd2b6e91224c6d7-5655134909e94716","baggage":"sentry-trace_id=aa766160c03649388cd2b6e91224c6d7,sentry-release=Unknown"},"correlation_id":"01KW9M363AQK5G59D1J857N27S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:035a6efbba73d972b95d6ede368f76db751e8dcce2608ce9681d49f2af7ffb3d","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:35.889Z","job_size_bytes":467,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-388ce0a5176053dfd857196b: start","job_status":"start","queue_duration_s":0.455165,"scheduling_latency_s":0.455197}1976vm-test-run-nixbot-gitlab> gitlab # [  306.636492] gitaly[1370]: time="2026-06-29T12:01:36.221Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1176 command.oublock=0 command.real_time_ms=26 command.spawn_token_fork_ms=3 command.system_time_ms=9 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-29T18:01:36.194" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:36.195" grpc.time_ms=25.519 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1977vm-test-run-nixbot-gitlab> gitlab # [  306.651668] gitaly[1370]: time="2026-06-29T12:01:36.318Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=592 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-29T12:02:06.200" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:36.300" grpc.time_ms=17.881 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1978vm-test-run-nixbot-gitlab> gitlab # [  306.665254] gitaly[1370]: time="2026-06-29T12:01:36.402Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:06.301" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:36.402" grpc.time_ms=0.68 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1979vm-test-run-nixbot-gitlab> gitlab # [  306.677721] gitaly[1370]: time="2026-06-29T12:01:36.422Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T12:01:46.095" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:36.405" grpc.time_ms=16.925 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1980vm-test-run-nixbot-gitlab> gitlab # [  306.746391] gitaly[1370]: time="2026-06-29T12:01:36.610Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T12:02:06.509" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:36.609" grpc.time_ms=1.092 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1981vm-test-run-nixbot-gitlab> gitlab # [  306.766096] gitaly[1370]: time="2026-06-29T12:01:36.624Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=29 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-29T12:02:06.519" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:36.619" grpc.time_ms=4.181 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1982vm-test-run-nixbot-gitlab> gitlab # [  306.786824] nixbot[3805]: INFO:     Started server process [3805]1983vm-test-run-nixbot-gitlab> gitlab # [  306.787759] nixbot[3805]: INFO:     Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1984vm-test-run-nixbot-gitlab> gitlab # [  306.789189] nixbot[3805]: INFO:     ::ffff:127.0.0.1:34100 - "GET /health HTTP/1.1" 200 OK1985vm-test-run-nixbot-gitlab> gitlab # [  306.793351] systemd[1]: Started nixbot CI service.1986vm-test-run-nixbot-gitlab> gitlab # [  306.800834] systemd[1]: Reached target Multi-User System.1987vm-test-run-nixbot-gitlab> gitlab # [  306.804360] systemd[1]: Startup finished in 809ms (kernel) + 5.104s (initrd) + 5min 890ms (userspace) = 5min 6.804s.1988vm-test-run-nixbot-gitlab> gitlab # [  306.885857] gitaly[1370]: time="2026-06-29T12:01:36.749Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T12:01:46.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:36.732" grpc.time_ms=17.648 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1989vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit nixbot.service, in 1.11 seconds)1990vm-test-run-nixbot-gitlab> gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1991vm-test-run-nixbot-gitlab> gitlab # [  307.359789] nixbot[3805]: INFO:     ::ffff:127.0.0.1:34110 - "GET /health HTTP/1.1" 200 OK1992vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.05 seconds)1993vm-test-run-nixbot-gitlab> subtest: project discovered and webhook registered1994vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1995vm-test-run-nixbot-gitlab> gitlab # [  307.430736] gitaly[1370]: time="2026-06-29T12:01:37.293Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=885 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=2 command.system_time_ms=18 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M38VJ9PXJNCDM7BWWS1BA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T12:02:07.066" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:37.267" grpc.time_ms=25.408 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1996vm-test-run-nixbot-gitlab> gitlab # [  307.610718] gitaly[1370]: time="2026-06-29T12:01:37.472Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=885 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=2 command.system_time_ms=20 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T12:02:07.345" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:37.446" grpc.time_ms=26.031 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1997vm-test-run-nixbot-gitlab> gitlab # [  307.659283] gitaly[1370]: time="2026-06-29T12:01:37.499Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M363AQK5G59D1J857N27S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-29T12:02:07.375" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:37.476" grpc.time_ms=23.173 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1998vm-test-run-nixbot-gitlab> gitlab # [  307.717465] sidekiq-cluster[2415]: {"severity":"WARN","time":"2026-06-29T12:01:37.580Z","class":"CreatePipelineWorker","correlation_id":"01KW9M363AQK5G59D1J857N27S","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"55d99a0d7974c5e56383b279aac3e27df907757a","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"55d99a0d7974c5e56383b279aac3e27df907757a","ref":"refs/heads/master"},"retry":0}1999vm-test-run-nixbot-gitlab> gitlab # [  307.738485] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:37.591Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"388ce0a5176053dfd857196b","created_at":"2026-06-29T12:01:35.875Z","trace_propagation_headers":{"sentry-trace":"aa766160c03649388cd2b6e91224c6d7-5655134909e94716","baggage":"sentry-trace_id=aa766160c03649388cd2b6e91224c6d7,sentry-release=Unknown"},"correlation_id":"01KW9M363AQK5G59D1J857N27S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:035a6efbba73d972b95d6ede368f76db751e8dcce2608ce9681d49f2af7ffb3d","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:35.889Z","job_size_bytes":467,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-388ce0a5176053dfd857196b: done: 1.247355 sec","job_status":"done","queue_duration_s":0.455165,"scheduling_latency_s":0.455197,"gitaly_calls":7,"gitaly_duration_s":0.111188,"redis_calls":53,"redis_duration_s":0.016406,"redis_read_bytes":171,"redis_write_bytes":4427,"redis_cache_calls":4,"redis_cache_duration_s":0.00368,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.007462,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001503,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002402,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001359,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":40,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":40,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.077,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.071,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.022906,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00047688,"duration_s":1.247355,"completed_at":"2026-06-29T12:01:37.591Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.112371,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2000vm-test-run-nixbot-gitlab> gitlab # [  307.825388] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:37.594Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"8efc62fbd93cf7859918b8fc","created_at":"2026-06-29T12:01:36.143Z","trace_propagation_headers":{"sentry-trace":"aa766160c03649388cd2b6e91224c6d7-29961cdf44f24d53","baggage":"sentry-trace_id=aa766160c03649388cd2b6e91224c6d7,sentry-release=Unknown"},"correlation_id":"01KW9M363AQK5G59D1J857N27S","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:36.148Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-8efc62fbd93cf7859918b8fc: start","job_status":"start","queue_duration_s":1.445424,"scheduling_latency_s":1.445454}2001vm-test-run-nixbot-gitlab> gitlab # [  307.847626] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:37.648Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"8efc62fbd93cf7859918b8fc","created_at":"2026-06-29T12:01:36.143Z","trace_propagation_headers":{"sentry-trace":"aa766160c03649388cd2b6e91224c6d7-29961cdf44f24d53","baggage":"sentry-trace_id=aa766160c03649388cd2b6e91224c6d7,sentry-release=Unknown"},"correlation_id":"01KW9M363AQK5G59D1J857N27S","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:36.148Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-8efc62fbd93cf7859918b8fc: done: 0.05447 sec","job_status":"done","queue_duration_s":1.445424,"scheduling_latency_s":1.445454,"redis_calls":19,"redis_duration_s":0.004953,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001039,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000613,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002394,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00860202,"duration_s":0.05447,"completed_at":"2026-06-29T12:01:37.648Z","load_balancing_strategy":"primary","db_duration_s":0.005858,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2002vm-test-run-nixbot-gitlab> gitlab # [  307.898838] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:37.650Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"f3cc87f31ed0e9187fe45565","created_at":"2026-06-29T12:01:37.633Z","trace_propagation_headers":{"sentry-trace":"aa766160c03649388cd2b6e91224c6d7-c7b3ed2738054d20","baggage":"sentry-trace_id=aa766160c03649388cd2b6e91224c6d7,sentry-release=Unknown"},"correlation_id":"01KW9M363AQK5G59D1J857N27S","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:37.639Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-f3cc87f31ed0e9187fe45565: start","job_status":"start","queue_duration_s":0.011003,"scheduling_latency_s":0.011024}2003vm-test-run-nixbot-gitlab> gitlab # [  307.916070] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:37.732Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"f3cc87f31ed0e9187fe45565","created_at":"2026-06-29T12:01:37.633Z","trace_propagation_headers":{"sentry-trace":"aa766160c03649388cd2b6e91224c6d7-c7b3ed2738054d20","baggage":"sentry-trace_id=aa766160c03649388cd2b6e91224c6d7,sentry-release=Unknown"},"correlation_id":"01KW9M363AQK5G59D1J857N27S","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:37.639Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-f3cc87f31ed0e9187fe45565: done: 0.081818 sec","job_status":"done","queue_duration_s":0.011003,"scheduling_latency_s":0.011024,"redis_calls":11,"redis_duration_s":0.002744,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000488,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001721,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000535,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.060263,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033018,"duration_s":0.081818,"completed_at":"2026-06-29T12:01:37.732Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016692,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2004vm-test-run-nixbot-gitlab> gitlab # [  307.961578] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:37 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 11862005vm-test-run-nixbot-gitlab> gitlab # [  307.964200] nixbot[3805]: {"ts": "2026-06-29T12:01:37.814Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}2006vm-test-run-nixbot-gitlab> gitlab # [  307.988188] nixbot[3805]: {"ts": "2026-06-29T12:01:37.852Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}2007vm-test-run-nixbot-gitlab> gitlab # [  308.056417] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:37 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 6472008vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.71 seconds)2009vm-test-run-nixbot-gitlab> []2010vm-test-run-nixbot-gitlab> gitlab # [  308.085435] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:37 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 842011vm-test-run-nixbot-gitlab> gitlab # [  308.088759] nixbot[3805]: {"ts": "2026-06-29T12:01:37.951Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}2012vm-test-run-nixbot-gitlab> gitlab # [  308.091927] nixbot[3805]: {"ts": "2026-06-29T12:01:37.954Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}2013vm-test-run-nixbot-gitlab> gitlab # [  308.350043] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 2542014vm-test-run-nixbot-gitlab> gitlab # [  308.352317] nixbot[3805]: {"ts": "2026-06-29T12:01:38.215Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}2015vm-test-run-nixbot-gitlab> gitlab # [  308.551959] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:38 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 1852016vm-test-run-nixbot-gitlab> gitlab # [  308.559151] nixbot[3805]: {"ts": "2026-06-29T12:01:38.416Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}2017vm-test-run-nixbot-gitlab> gitlab # [  308.816376] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 2552018vm-test-run-nixbot-gitlab> gitlab # [  308.818702] nixbot[3805]: {"ts": "2026-06-29T12:01:38.682Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}2019vm-test-run-nixbot-gitlab> gitlab # [  308.971998] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:38 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 1442020vm-test-run-nixbot-gitlab> gitlab # [  308.979702] nixbot[3805]: {"ts": "2026-06-29T12:01:38.836Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}2021vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks2022vm-test-run-nixbot-gitlab> gitlab # [  309.124850] gitaly[1370]: time="2026-06-29T12:01:38.987Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3B10VZV9XBFCZCXMKM0H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-06-29T12:02:08.666" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:38.967" grpc.time_ms=19.829 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2023vm-test-run-nixbot-gitlab> gitlab # [  309.161890] gitaly[1370]: time="2026-06-29T12:01:39.025Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3B10VZV9XBFCZCXMKM0H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:09.023" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:39.024" grpc.time_ms=0.965 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2024vm-test-run-nixbot-gitlab> gitlab # [  309.195497] gitaly[1370]: time="2026-06-29T12:01:39.048Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3B10VZV9XBFCZCXMKM0H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-29T12:01:49.000" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:39.031" grpc.time_ms=16.457 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2025vm-test-run-nixbot-gitlab> gitlab # [  309.229280] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 1072026vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.16 seconds)2027vm-test-run-nixbot-gitlab> [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-06-29T12:01:38.198Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2028vm-test-run-nixbot-gitlab> (finished: subtest: project discovered and webhook registered, in 1.87 seconds)2029vm-test-run-nixbot-gitlab> subtest: push delivers the webhook2030vm-test-run-nixbot-gitlab> gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master2031vm-test-run-nixbot-gitlab> gitlab # [  309.301108] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 3162032vm-test-run-nixbot-gitlab> gitlab # [  309.306621] nixbot[3805]: {"ts": "2026-06-29T12:01:39.167Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}2033vm-test-run-nixbot-gitlab> gitlab # [  309.415291] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:39 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 732034vm-test-run-nixbot-gitlab> gitlab # [  309.535929] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 2142035vm-test-run-nixbot-gitlab> gitlab # [  309.543244] nixbot[3805]: {"ts": "2026-06-29T12:01:39.396Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}2036vm-test-run-nixbot-gitlab> gitlab # [  309.550540] nixbot[3805]: {"ts": "2026-06-29T12:01:39.404Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}2037vm-test-run-nixbot-gitlab> gitlab # [  309.783948] gitaly[1370]: time="2026-06-29T12:01:39.645Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BERD53BJKME5BDBSSZN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:39.622" grpc.time_ms=22.432 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root2038vm-test-run-nixbot-gitlab> gitlab # Enumerating objects: 5, done.2039vm-test-run-nixbot-gitlab> gitlab # [  309.809263] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:39 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.54.0" 3582040vm-test-run-nixbot-gitlab> gitlab # Counting objects:  20% (1/5)Counting objects:  40% (2/5)Counting objects:  60% (3/5)Counting objects:  80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.2041vm-test-run-nixbot-gitlab> gitlab # Delta compression using up to 4 threads2042vm-test-run-nixbot-gitlab> gitlab # Compressing objects:  50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.2043vm-test-run-nixbot-gitlab> gitlab # [  309.822246] gitaly[1370]: time="2026-06-29T12:01:39.686Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=5 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BMR1ZJQ63MGWHN9B4EJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:39.673" grpc.time_ms=13.203 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc2044vm-test-run-nixbot-gitlab> gitlab # [  309.836468] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 2052045vm-test-run-nixbot-gitlab> gitlab # Writing objects:  33% (1/3)Writing objects:  66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 16.00 KiB/s, done.2046vm-test-run-nixbot-gitlab> gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)2047vm-test-run-nixbot-gitlab> gitlab # [  309.971884] gitaly[1370]: time="2026-06-29T12:01:39.836Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3BV7YS3HZHX8KNESHS6K grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:39.827" pid=1370 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc2048vm-test-run-nixbot-gitlab> gitlab # [  309.983756] gitaly[1370]: time="2026-06-29T12:01:39.836Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3BV7YS3HZHX8KNESHS6K grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:39.827" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc2049vm-test-run-nixbot-gitlab> gitlab # [  309.998390] gitlab-workhorse[1286]: time="2026-06-29T12:01:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1492050vm-test-run-nixbot-gitlab> gitlab # [  310.000930] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 1482051vm-test-run-nixbot-gitlab> gitlab # [  310.002953] gitaly[1370]: time="2026-06-29T12:01:39.836Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3BV7YS3HZHX8KNESHS6K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:39.827" grpc.time_ms=8.898 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc2052vm-test-run-nixbot-gitlab> gitlab # [  310.088325] gitaly[1370]: time="2026-06-29T12:01:39.952Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3C0GXVDZHACCX8TCNQVR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:39.929" pid=1370 remote_ip="2001:db8:1::1" request_sha=fdbc2c3557a50562063cf90ac2d8d30811288120 response_bytes=373 span.kind=server system=grpc2053vm-test-run-nixbot-gitlab> gitlab # [  310.099865] gitaly[1370]: time="2026-06-29T12:01:39.952Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1162 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=6 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3C0GXVDZHACCX8TCNQVR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:39.929" negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc2054vm-test-run-nixbot-gitlab> gitlab # [  310.114758] gitlab-workhorse[1286]: time="2026-06-29T12:01:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3732055vm-test-run-nixbot-gitlab> gitlab # [  310.116778] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 373 "" "git/2.54.0" 962056vm-test-run-nixbot-gitlab> gitlab # [  310.118729] gitaly[1370]: time="2026-06-29T12:01:39.952Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1162 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=6 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3C0GXVDZHACCX8TCNQVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=9 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:39.929" grpc.time_ms=23.179 negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc2057vm-test-run-nixbot-gitlab> gitlab # [  310.164413] redis[1252]: 10 changes in 300 seconds. Saving...2058vm-test-run-nixbot-gitlab> gitlab # [  310.165533] redis[1252]: Background saving started by pid 39802059vm-test-run-nixbot-gitlab> gitlab # [  310.167787] redis[3980]: BGSAVE done, 577 keys saved, 0 keys skipped, 135499 bytes written.2060vm-test-run-nixbot-gitlab> gitlab # [  310.171208] gitaly[1370]: time="2026-06-29T12:01:40.034Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=589 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=5 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-29T12:01:50.000" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:40.021" grpc.time_ms=13.631 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2061vm-test-run-nixbot-gitlab> gitlab # [  310.185603] redis[3980]: DB saved on disk2062vm-test-run-nixbot-gitlab> gitlab # [  310.186411] redis[3980]: Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB2063vm-test-run-nixbot-gitlab> gitlab # [  310.236457] gitaly[1370]: time="2026-06-29T12:01:40.100Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3C5BP926ZMW21BSH5B5H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:40.088" grpc.time_ms=11.674 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc2064vm-test-run-nixbot-gitlab> gitlab # [  310.250422] gitaly[1370]: time="2026-06-29T12:01:40.103Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-29T12:02:10.093" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-29T12:01:40.093" grpc.time_ms=9.632 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2065vm-test-run-nixbot-gitlab> gitlab # [  310.266164] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:40 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 892066vm-test-run-nixbot-gitlab> gitlab # [  310.268489] redis[1252]: Background saving terminated with success2067vm-test-run-nixbot-gitlab> gitlab # [  310.280245] gitaly[1370]: time="2026-06-29T12:01:40.144Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J duration_ms=205ns method=POST pid=1370 status=200 url="http://unix/api/v4/internal/allowed"2068vm-test-run-nixbot-gitlab> gitlab # [  310.309886] gitaly[1370]: time="2026-06-29T12:01:40.172Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J duration_ms=24ns method=POST pid=1370 status=200 url="http://unix/api/v4/internal/pre_receive"2069vm-test-run-nixbot-gitlab> gitlab # [  310.313165] gitaly[1370]: time="2026-06-29T12:01:40.173Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5318 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:39.938" grpc.time_ms=234.056 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root2070vm-test-run-nixbot-gitlab> gitlab # [  310.323922] gitaly[1370]: time="2026-06-29T12:01:40.180Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3C87WXWYTE882Z9T75YB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:40.171" pid=1370 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc2071vm-test-run-nixbot-gitlab> gitlab # [  310.334690] gitlab-workhorse[1286]: time="2026-06-29T12:01:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672072vm-test-run-nixbot-gitlab> gitlab # [  310.336696] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 842073vm-test-run-nixbot-gitlab> gitlab # [  310.339210] gitaly[1370]: time="2026-06-29T12:01:40.180Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=5 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3C87WXWYTE882Z9T75YB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:40.171" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc2074vm-test-run-nixbot-gitlab> gitlab # [  310.353404] gitaly[1370]: time="2026-06-29T12:01:40.180Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=5 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3C87WXWYTE882Z9T75YB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:40.171" grpc.time_ms=8.465 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc2075vm-test-run-nixbot-gitlab> gitlab # [  310.368419] gitaly[1370]: time="2026-06-29T12:01:40.209Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4788 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:40.209" grpc.time_ms=0.447 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root2076vm-test-run-nixbot-gitlab> gitlab # [  310.378856] gitaly[1370]: time="2026-06-29T12:01:40.238Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:40.237" grpc.time_ms=0.875 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root2077vm-test-run-nixbot-gitlab> gitlab # [  310.397522] gitaly[1370]: time="2026-06-29T12:01:40.261Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4789 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:40.261" grpc.time_ms=0.454 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root2078vm-test-run-nixbot-gitlab> gitlab # [  310.434127] gitaly[1370]: time="2026-06-29T12:01:40.298Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3CC34F8J33QX6F9V2H5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:10.097" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:40.297" grpc.time_ms=0.423 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2079vm-test-run-nixbot-gitlab> gitlab # [  310.468320] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:40.330Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","55d99a0d7974c5e56383b279aac3e27df907757a 084809b87ce4cd362622ba41e849f7e129b03d15 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"b2f08d1ad2dc07b1c7962063","created_at":"2026-06-29T12:01:40.325Z","trace_propagation_headers":{"sentry-trace":"f40b0a3c6a1e4971b7234644bbdbda88-89896708b3b14573","baggage":"sentry-trace_id=f40b0a3c6a1e4971b7234644bbdbda88,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3BVWWCP2RZMADEK6NG0J","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:40.328Z","job_size_bytes":185,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-b2f08d1ad2dc07b1c7962063: start","job_status":"start","queue_duration_s":0.001707,"scheduling_latency_s":0.001753}2080vm-test-run-nixbot-gitlab> gitlab # [  310.493684] gitaly[1370]: time="2026-06-29T12:01:40.357Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=592 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3CC34F8J33QX6F9V2H5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-29T12:02:10.046" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:40.346" grpc.time_ms=10.731 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2081vm-test-run-nixbot-gitlab> gitlab # [  310.510035] gitaly[1370]: time="2026-06-29T12:01:40.370Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J duration_ms=75ns method=POST pid=1370 status=200 url="http://unix/api/v4/internal/post_receive"2082vm-test-run-nixbot-gitlab> gitlab # [  310.514471] gitaly[1370]: time="2026-06-29T12:01:40.372Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4812 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:40.294" grpc.time_ms=77.237 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root2083vm-test-run-nixbot-gitlab> gitlab # To http://gitlab/root/test-flake.git2084vm-test-run-nixbot-gitlab> gitlab # [  310.525813] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:40.360Z","message":"changes: \"55d99a0d7974c5e56383b279aac3e27df907757a 084809b87ce4cd362622ba41e849f7e129b03d15 refs/heads/master\\n\""}2085vm-test-run-nixbot-gitlab> gitlab #    55d99a0..084809b  [  310.528685] gitlab-workhorse[1286]: time="2026-06-29T12:01:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=522086vm-test-run-nixbot-gitlab> gitlab # master -> master2087vm-test-run-nixbot-gitlab> gitlab # [  310.531826] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:40 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 6792088vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.32 seconds)2089vm-test-run-nixbot-gitlab> gitlab: must succeed: git -C /tmp/test-flake rev-parse master2090vm-test-run-nixbot-gitlab> gitlab # [  310.535072] gitaly[1370]: time="2026-06-29T12:01:40.386Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J diskcache=d4aab82d-2f5d-4766-a832-5540ddc85bb3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:39.812" pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root2091vm-test-run-nixbot-gitlab> gitlab # [  310.545338] gitaly[1370]: time="2026-06-29T12:01:40.386Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=142 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=12918 command.oublock=32 command.real_time_ms=565 command.spawn_token_fork_ms=5 command.system_time_ms=70 command.user_time_ms=72 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:39.812" grpc.time_ms=573.699 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root2092vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds)2093vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=084809b87ce4cd362622ba41e849f7e129b03d15'2094vm-test-run-nixbot-gitlab> gitlab # [  310.558790] gitaly[1370]: time="2026-06-29T12:01:40.393Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=6 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3CC34F8J33QX6F9V2H5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T12:01:50.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:40.375" grpc.time_ms=18.188 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2095vm-test-run-nixbot-gitlab> gitlab # [  310.572938] gitaly[1370]: time="2026-06-29T12:01:40.396Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T12:01:50.073" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:40.383" grpc.time_ms=13.214 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2096vm-test-run-nixbot-gitlab> gitlab # [  310.586286] gitaly[1370]: time="2026-06-29T12:01:40.409Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T12:01:50.091" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:40.402" grpc.time_ms=6.858 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2097vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=084809b87ce4cd362622ba41e849f7e129b03d15', in 0.03 seconds)2098vm-test-run-nixbot-gitlab> []2099vm-test-run-nixbot-gitlab> gitlab # [  310.600237] gitaly[1370]: time="2026-06-29T12:01:40.411Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3CC34F8J33QX6F9V2H5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T12:01:50.033" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:40.403" grpc.time_ms=7.618 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2100vm-test-run-nixbot-gitlab> gitlab # [  310.613907] nixbot[3805]: INFO:     ::ffff:127.0.0.1:34116 - "GET /api/repos/gitlab/root/test-flake/builds?commit=084809b87ce4cd362622ba41e849f7e129b03d15 HTTP/1.1" 200 OK2101vm-test-run-nixbot-gitlab> gitlab # [  310.615897] gitaly[1370]: time="2026-06-29T12:01:40.438Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-29T12:01:50.119" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:40.430" grpc.time_ms=7.884 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2102vm-test-run-nixbot-gitlab> gitlab # [  310.629842] gitaly[1370]: time="2026-06-29T12:01:40.454Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=599 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-06-29T12:02:10.344" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:40.445" grpc.time_ms=9.385 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2103vm-test-run-nixbot-gitlab> gitlab # [  310.645610] gitaly[1370]: time="2026-06-29T12:01:40.467Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=319 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-29T12:01:50.152" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-06-29T12:01:40.463" grpc.time_ms=4.776 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2104vm-test-run-nixbot-gitlab> gitlab # [  310.660146] gitaly[1370]: time="2026-06-29T12:01:40.483Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T12:01:50.165" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:40.476" grpc.time_ms=7.62 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2105vm-test-run-nixbot-gitlab> gitlab # [  310.674755] gitaly[1370]: time="2026-06-29T12:01:40.484Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3CC34F8J33QX6F9V2H5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T12:01:50.114" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:40.484" grpc.time_ms=0.243 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2106vm-test-run-nixbot-gitlab> gitlab # [  310.686372] gitaly[1370]: time="2026-06-29T12:01:40.501Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3CC34F8J33QX6F9V2H5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T12:01:50.121" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:40.491" grpc.time_ms=9.528 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2107vm-test-run-nixbot-gitlab> gitlab # [  310.700599] gitaly[1370]: time="2026-06-29T12:01:40.533Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M3CC34F8J33QX6F9V2H5S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:40.532" grpc.time_ms=0.359 pid=1370 span.kind=server system=grpc2108vm-test-run-nixbot-gitlab> gitlab # [  310.710729] gitaly[1370]: time="2026-06-29T12:01:40.553Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M3CC34F8J33QX6F9V2H5S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4157 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:40.553" grpc.time_ms=0.369 pid=1370 span.kind=server system=grpc2109vm-test-run-nixbot-gitlab> gitlab # [  310.720711] gitaly[1370]: time="2026-06-29T12:01:40.560Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3CC34F8J33QX6F9V2H5S diskcache=bf36c34c-f04f-4545-b82f-c1cb3059bdc9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T12:01:50.132" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:40.502" pid=1370 ref=refs/keep-around/55d99a0d7974c5e56383b279aac3e27df907757a remote_ip="2001:db8:1::1" revision=55d99a0d7974c5e56383b279aac3e27df907757a span.kind=server system=grpc username=root2110vm-test-run-nixbot-gitlab> gitlab # [  310.732214] gitaly[1370]: time="2026-06-29T12:01:40.561Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=50 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5110 command.oublock=8 command.real_time_ms=52 command.spawn_token_fork_ms=0 command.system_time_ms=39 command.user_time_ms=11 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3CC34F8J33QX6F9V2H5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T12:01:50.132" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:40.502" grpc.time_ms=58.195 pid=1370 ref=refs/keep-around/55d99a0d7974c5e56383b279aac3e27df907757a remote_ip="2001:db8:1::1" revision=55d99a0d7974c5e56383b279aac3e27df907757a span.kind=server system=grpc username=root2111vm-test-run-nixbot-gitlab> gitlab # [  310.747692] gitaly[1370]: time="2026-06-29T12:01:40.564Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3CC34F8J33QX6F9V2H5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:10.363" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:40.563" grpc.time_ms=0.229 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2112vm-test-run-nixbot-gitlab> gitlab # [  310.831107] gitaly[1370]: time="2026-06-29T12:01:40.695Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-29T12:01:50.002" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:40.693" grpc.time_ms=1.601 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2113vm-test-run-nixbot-gitlab> gitlab # [  310.858001] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:40.716Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","55d99a0d7974c5e56383b279aac3e27df907757a 084809b87ce4cd362622ba41e849f7e129b03d15 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"b2f08d1ad2dc07b1c7962063","created_at":"2026-06-29T12:01:40.325Z","trace_propagation_headers":{"sentry-trace":"f40b0a3c6a1e4971b7234644bbdbda88-89896708b3b14573","baggage":"sentry-trace_id=f40b0a3c6a1e4971b7234644bbdbda88,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3BVWWCP2RZMADEK6NG0J","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T12:01:40.328Z","job_size_bytes":185,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-b2f08d1ad2dc07b1c7962063: done: 0.385463 sec","job_status":"done","queue_duration_s":0.001707,"scheduling_latency_s":0.001753,"gitaly_calls":7,"gitaly_duration_s":0.069592,"redis_calls":108,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.026217,"redis_read_bytes":349,"redis_write_bytes":12823,"redis_cache_calls":5,"redis_cache_duration_s":0.002635,"redis_cache_read_bytes":13,"redis_cache_write_bytes":347,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001664,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.001142,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6769,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.002192,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.012216,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.006368,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.023,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.023,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.253766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00069834,"exclusive_lock_hold_duration_s":0.0272351,"duration_s":0.385463,"completed_at":"2026-06-29T12:01:40.715Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.037659,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2114vm-test-run-nixbot-gitlab> gitlab # [  310.931418] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:40.718Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"8bf3f2f8888c1eb8e4051fb0","created_at":"2026-06-29T12:01:40.412Z","trace_propagation_headers":{"sentry-trace":"f40b0a3c6a1e4971b7234644bbdbda88-40d6ae6ad0f54311","baggage":"sentry-trace_id=f40b0a3c6a1e4971b7234644bbdbda88,sentry-release=Unknown"},"correlation_id":"01KW9M3BVWWCP2RZMADEK6NG0J","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:40.414Z","job_size_bytes":46,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8bf3f2f8888c1eb8e4051fb0: start","job_status":"start","queue_duration_s":0.303551,"scheduling_latency_s":0.303583}2115vm-test-run-nixbot-gitlab> gitlab # [  310.949867] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:40.743Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"8bf3f2f8888c1eb8e4051fb0","created_at":"2026-06-29T12:01:40.412Z","trace_propagation_headers":{"sentry-trace":"f40b0a3c6a1e4971b7234644bbdbda88-40d6ae6ad0f54311","baggage":"sentry-trace_id=f40b0a3c6a1e4971b7234644bbdbda88,sentry-release=Unknown"},"correlation_id":"01KW9M3BVWWCP2RZMADEK6NG0J","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:40.414Z","job_size_bytes":46,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8bf3f2f8888c1eb8e4051fb0: done: 0.025676 sec","job_status":"done","queue_duration_s":0.303551,"scheduling_latency_s":0.303583,"redis_calls":9,"redis_duration_s":0.002695,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00114,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000311,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001244,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014868,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025676,"completed_at":"2026-06-29T12:01:40.743Z","load_balancing_strategy":"primary","db_duration_s":0.002751,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2116vm-test-run-nixbot-gitlab> gitlab # [  310.996181] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:40.744Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"46f462abd58b02324a77a3cd","created_at":"2026-06-29T12:01:40.420Z","trace_propagation_headers":{"sentry-trace":"f40b0a3c6a1e4971b7234644bbdbda88-40d6ae6ad0f54311","baggage":"sentry-trace_id=f40b0a3c6a1e4971b7234644bbdbda88,sentry-release=Unknown"},"correlation_id":"01KW9M3BVWWCP2RZMADEK6NG0J","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:40.422Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-46f462abd58b02324a77a3cd: start","job_status":"start","queue_duration_s":0.322137,"scheduling_latency_s":0.322157}2117vm-test-run-nixbot-gitlab> gitlab # [  311.012875] gitaly[1370]: time="2026-06-29T12:01:40.798Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J diskcache=9b056bfc-dd75-4979-b76e-fc9948a408ff grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-29T18:01:40.768" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:40.769" pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2118vm-test-run-nixbot-gitlab> gitlab # [  311.023004] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/55d99a0d7974c5e56383b279aac3e27df907757a HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 5102119vm-test-run-nixbot-gitlab> gitlab # [  311.025682] nixbot[3805]: {"ts": "2026-06-29T12:01:40.739Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/55d99a0d7974c5e56383b279aac3e27df907757a \"HTTP/1.1 201 Created\""}2120vm-test-run-nixbot-gitlab> gitlab # [  311.028488] nixbot[3805]: {"ts": "2026-06-29T12:01:40.753Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-55d99a0d7974-72e412b4", "argv0": "bwrap"}2121vm-test-run-nixbot-gitlab> gitlab # [  311.030945] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:40.826Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"46f462abd58b02324a77a3cd","created_at":"2026-06-29T12:01:40.420Z","trace_propagation_headers":{"sentry-trace":"f40b0a3c6a1e4971b7234644bbdbda88-40d6ae6ad0f54311","baggage":"sentry-trace_id=f40b0a3c6a1e4971b7234644bbdbda88,sentry-release=Unknown"},"correlation_id":"01KW9M3BVWWCP2RZMADEK6NG0J","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:40.422Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-46f462abd58b02324a77a3cd: done: 0.082032 sec","job_status":"done","queue_duration_s":0.322137,"scheduling_latency_s":0.322157,"gitaly_calls":1,"gitaly_duration_s":0.032064,"redis_calls":12,"redis_duration_s":0.001963,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000847,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000326,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00079,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.013,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.013,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.057378,"duration_s":0.082032,"completed_at":"2026-06-29T12:01:40.826Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019441,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2122vm-test-run-nixbot-gitlab> gitlab # [  311.077543] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:40.828Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2d0125217749e30b14798866","created_at":"2026-06-29T12:01:40.529Z","trace_propagation_headers":{"sentry-trace":"f40b0a3c6a1e4971b7234644bbdbda88-40d6ae6ad0f54311","baggage":"sentry-trace_id=f40b0a3c6a1e4971b7234644bbdbda88,sentry-release=Unknown"},"correlation_id":"01KW9M3BVWWCP2RZMADEK6NG0J","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:39eff73a01812350421f8350f6478b98645ab901b96da7c87ee4df78efd8c8a3","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:40.531Z","job_size_bytes":467,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2d0125217749e30b14798866: start","job_status":"start","queue_duration_s":0.29663,"scheduling_latency_s":0.29665}2123vm-test-run-nixbot-gitlab> gitlab # [  311.095371] gitaly[1370]: time="2026-06-29T12:01:40.798Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1509 command.oublock=0 command.real_time_ms=30 command.spawn_token_fork_ms=3 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-29T18:01:40.768" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:40.769" grpc.time_ms=29.194 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2124vm-test-run-nixbot-gitlab> gitlab # [  311.110996] gitaly[1370]: time="2026-06-29T12:01:40.854Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:10.753" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:40.853" grpc.time_ms=0.584 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2125vm-test-run-nixbot-gitlab> gitlab # [  311.123785] gitaly[1370]: time="2026-06-29T12:01:40.865Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T12:01:50.126" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:40.856" grpc.time_ms=8.301 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2126vm-test-run-nixbot-gitlab> gitlab # [  311.137442] systemd[1]: Started Nix Daemon.2127vm-test-run-nixbot-gitlab> gitlab # [  311.138107] gitaly[1370]: time="2026-06-29T12:01:40.904Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T12:02:10.802" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:40.903" grpc.time_ms=1.305 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2128vm-test-run-nixbot-gitlab> gitlab # [  311.150840] gitaly[1370]: time="2026-06-29T12:01:40.917Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=195 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-29T12:02:10.808" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:40.908" grpc.time_ms=7.829 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2129vm-test-run-nixbot-gitlab> gitlab # [  311.167155] gitaly[1370]: time="2026-06-29T12:01:40.971Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T12:01:50.233" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:40.963" grpc.time_ms=8.07 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2130vm-test-run-nixbot-gitlab> gitlab # [  311.275800] nix-daemon[4097]: accepted connection from pid 4093, user nixbot2131vm-test-run-nixbot-gitlab> gitlab # [  311.280732] nix-daemon[4097]: accepted connection from pid 4092, user nixbot2132vm-test-run-nixbot-gitlab> gitlab # [  311.374984] postgres[1324]: [1324] LOG:  checkpoint starting: time2133vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=084809b87ce4cd362622ba41e849f7e129b03d15'2134vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=084809b87ce4cd362622ba41e849f7e129b03d15', in 0.08 seconds)2135vm-test-run-nixbot-gitlab> []2136vm-test-run-nixbot-gitlab> gitlab # [  311.668171] gitaly[1370]: time="2026-06-29T12:01:41.530Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=884 command.oublock=0 command.real_time_ms=26 command.spawn_token_fork_ms=4 command.system_time_ms=10 command.user_time_ms=10 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T12:02:11.398" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:41.499" grpc.time_ms=31.284 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2137vm-test-run-nixbot-gitlab> gitlab # [  311.713145] gitaly[1370]: time="2026-06-29T12:01:41.561Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=3 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3BVWWCP2RZMADEK6NG0J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-29T12:02:11.433" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:41.534" grpc.time_ms=26.66 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2138vm-test-run-nixbot-gitlab> gitlab # [  311.739410] nixbot[3805]: INFO:     ::ffff:127.0.0.1:42868 - "GET /api/repos/gitlab/root/test-flake/builds?commit=084809b87ce4cd362622ba41e849f7e129b03d15 HTTP/1.1" 200 OK2139vm-test-run-nixbot-gitlab> gitlab # [  311.743187] sidekiq-cluster[2415]: {"severity":"WARN","time":"2026-06-29T12:01:41.586Z","class":"CreatePipelineWorker","correlation_id":"01KW9M3BVWWCP2RZMADEK6NG0J","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"55d99a0d7974c5e56383b279aac3e27df907757a","after":"084809b87ce4cd362622ba41e849f7e129b03d15","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"084809b87ce4cd362622ba41e849f7e129b03d15","ref":"refs/heads/master"},"retry":0}2140vm-test-run-nixbot-gitlab> gitlab # [  311.753871] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:41.597Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2d0125217749e30b14798866","created_at":"2026-06-29T12:01:40.529Z","trace_propagation_headers":{"sentry-trace":"f40b0a3c6a1e4971b7234644bbdbda88-40d6ae6ad0f54311","baggage":"sentry-trace_id=f40b0a3c6a1e4971b7234644bbdbda88,sentry-release=Unknown"},"correlation_id":"01KW9M3BVWWCP2RZMADEK6NG0J","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:39eff73a01812350421f8350f6478b98645ab901b96da7c87ee4df78efd8c8a3","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:40.531Z","job_size_bytes":467,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2d0125217749e30b14798866: done: 0.769043 sec","job_status":"done","queue_duration_s":0.29663,"scheduling_latency_s":0.29665,"gitaly_calls":7,"gitaly_duration_s":0.102953,"redis_calls":33,"redis_duration_s":0.005252,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000562,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001335,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001846,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001509,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.640281,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.769043,"completed_at":"2026-06-29T12:01:41.597Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026245,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2141vm-test-run-nixbot-gitlab> gitlab # [  311.806988] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:41.600Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"bfd6731bc70e96c588abe140","created_at":"2026-06-29T12:01:40.570Z","trace_propagation_headers":{"sentry-trace":"51d35019551a421086939d99051becd1-6f77540e4680435b","baggage":"sentry-trace_id=51d35019551a421086939d99051becd1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3CC34F8J33QX6F9V2H5S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:40.576Z","job_size_bytes":65,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-bfd6731bc70e96c588abe140: start","job_status":"start","queue_duration_s":1.023795,"scheduling_latency_s":1.023841}2142vm-test-run-nixbot-gitlab> gitlab # [  311.829894] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:41.693Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"bfd6731bc70e96c588abe140","created_at":"2026-06-29T12:01:40.570Z","trace_propagation_headers":{"sentry-trace":"51d35019551a421086939d99051becd1-6f77540e4680435b","baggage":"sentry-trace_id=51d35019551a421086939d99051becd1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3CC34F8J33QX6F9V2H5S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:40.576Z","job_size_bytes":65,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-bfd6731bc70e96c588abe140: done: 0.0929 sec","job_status":"done","queue_duration_s":1.023795,"scheduling_latency_s":1.023841,"redis_calls":18,"redis_duration_s":0.005938000000000001,"redis_read_bytes":271,"redis_write_bytes":2103,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.003215,"redis_feature_flag_read_bytes":264,"redis_feature_flag_write_bytes":1285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002027,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000696,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0006924,"duration_s":0.0929,"completed_at":"2026-06-29T12:01:41.693Z","load_balancing_strategy":"primary","db_duration_s":0.021356,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2143vm-test-run-nixbot-gitlab> gitlab # [  311.874605] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:41.695Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"71287a7661dd658a19a91789","created_at":"2026-06-29T12:01:40.577Z","trace_propagation_headers":{"sentry-trace":"51d35019551a421086939d99051becd1-6f77540e4680435b","baggage":"sentry-trace_id=51d35019551a421086939d99051becd1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3CC34F8J33QX6F9V2H5S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:40.581Z","job_size_bytes":65,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-71287a7661dd658a19a91789: start","job_status":"start","queue_duration_s":1.113888,"scheduling_latency_s":1.113919}2144vm-test-run-nixbot-gitlab> gitlab # [  311.987064] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:41.849Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"71287a7661dd658a19a91789","created_at":"2026-06-29T12:01:40.577Z","trace_propagation_headers":{"sentry-trace":"51d35019551a421086939d99051becd1-6f77540e4680435b","baggage":"sentry-trace_id=51d35019551a421086939d99051becd1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3CC34F8J33QX6F9V2H5S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:40.581Z","job_size_bytes":65,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-71287a7661dd658a19a91789: done: 0.154176 sec","job_status":"done","queue_duration_s":1.113888,"scheduling_latency_s":1.113919,"redis_calls":21,"redis_duration_s":0.010542,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.004354,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003397,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002791,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.112613,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00106024,"duration_s":0.154176,"completed_at":"2026-06-29T12:01:41.849Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026124,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2145vm-test-run-nixbot-gitlab> gitlab # [  312.075079] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:41.854Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"13fbe39a-596b-4323-abef-4daa0a58d721\"}"],"class":"WebHookWorker","jid":"ea74f6434df1abf6c443a28e","created_at":"2026-06-29T12:01:40.669Z","trace_propagation_headers":{"sentry-trace":"f40b0a3c6a1e4971b7234644bbdbda88-40d6ae6ad0f54311","baggage":"sentry-trace_id=f40b0a3c6a1e4971b7234644bbdbda88,sentry-release=Unknown"},"correlation_id":"01KW9M3BVWWCP2RZMADEK6NG0J","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b17e0e277d6f7081cdf0fd71720074b67d9896cead57902189c21fffaf13a4e","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:40.679Z","job_size_bytes":1749,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-ea74f6434df1abf6c443a28e: start","job_status":"start","queue_duration_s":1.17454,"scheduling_latency_s":1.174592}2146vm-test-run-nixbot-gitlab> gitlab # [  312.154907] nixbot[3805]: INFO:     ::ffff:127.0.0.1:42884 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2147vm-test-run-nixbot-gitlab> gitlab # [  312.194479] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:42.056Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"13fbe39a-596b-4323-abef-4daa0a58d721\"}"],"class":"WebHookWorker","jid":"ea74f6434df1abf6c443a28e","created_at":"2026-06-29T12:01:40.669Z","trace_propagation_headers":{"sentry-trace":"f40b0a3c6a1e4971b7234644bbdbda88-40d6ae6ad0f54311","baggage":"sentry-trace_id=f40b0a3c6a1e4971b7234644bbdbda88,sentry-release=Unknown"},"correlation_id":"01KW9M3BVWWCP2RZMADEK6NG0J","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b17e0e277d6f7081cdf0fd71720074b67d9896cead57902189c21fffaf13a4e","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:40.679Z","job_size_bytes":1749,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-ea74f6434df1abf6c443a28e: done: 0.202883 sec","job_status":"done","queue_duration_s":1.17454,"scheduling_latency_s":1.174592,"redis_calls":19,"redis_duration_s":0.007802,"redis_read_bytes":31,"redis_write_bytes":5282,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002006,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000719,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3644,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00327,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001807,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.014829039999995075,"cpu_s":0.163335,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000773,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.202883,"completed_at":"2026-06-29T12:01:42.056Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004205,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2148vm-test-run-nixbot-gitlab> gitlab # [  312.283039] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:42.060Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"810e06a475f7b12342fdb3d8","created_at":"2026-06-29T12:01:40.706Z","trace_propagation_headers":{"sentry-trace":"51d35019551a421086939d99051becd1-6f77540e4680435b","baggage":"sentry-trace_id=51d35019551a421086939d99051becd1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3CC34F8J33QX6F9V2H5S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:40.715Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-810e06a475f7b12342fdb3d8: start","job_status":"start","queue_duration_s":1.345305,"scheduling_latency_s":1.345347}2149vm-test-run-nixbot-gitlab> gitlab # [  312.303296] gitaly[1370]: time="2026-06-29T12:01:42.167Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3E5QZDK5GS2VSTR1YJKN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:42.158" grpc.time_ms=8.7 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc2150vm-test-run-nixbot-gitlab> gitlab # [  312.325258] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:42 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 972151vm-test-run-nixbot-gitlab> gitlab # [  312.368232] gitaly[1370]: time="2026-06-29T12:01:42.232Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3E8S4MSA3DFVAQ3TNM74 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:42.218" pid=1370 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc2152vm-test-run-nixbot-gitlab> gitlab # [  312.389448] gitaly[1370]: time="2026-06-29T12:01:42.232Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=588 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3E8S4MSA3DFVAQ3TNM74 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:42.218" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc2153vm-test-run-nixbot-gitlab> gitlab # [  312.410817] gitlab-workhorse[1286]: time="2026-06-29T12:01:42Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672154vm-test-run-nixbot-gitlab> gitlab # [  312.413433] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:42 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 632155vm-test-run-nixbot-gitlab> gitlab # [  312.415925] gitaly[1370]: time="2026-06-29T12:01:42.232Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=588 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3E8S4MSA3DFVAQ3TNM74 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:42.218" grpc.time_ms=14.041 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc2156vm-test-run-nixbot-gitlab> gitlab # [  312.460108] gitaly[1370]: time="2026-06-29T12:01:42.323Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3EAVMBD9KFVHXT0EKPRY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:42.289" pid=1370 remote_ip="2001:db8:1::1" request_sha=4656510e6c80ccf6ce1dd21bec2bf61955eb8542 response_bytes=384 span.kind=server system=grpc2157vm-test-run-nixbot-gitlab> gitlab # [  312.472056] gitlab-workhorse[1286]: time="2026-06-29T12:01:42Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3842158vm-test-run-nixbot-gitlab> gitlab # [  312.474219] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:42 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 384 "" "git/2.54.0" 882159vm-test-run-nixbot-gitlab> gitlab # [  312.476384] gitaly[1370]: time="2026-06-29T12:01:42.323Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1140 command.oublock=0 command.real_time_ms=31 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3EAVMBD9KFVHXT0EKPRY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:42.289" negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc2160vm-test-run-nixbot-gitlab> gitlab # [  312.490441] gitaly[1370]: time="2026-06-29T12:01:42.323Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1140 command.oublock=0 command.real_time_ms=31 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3EAVMBD9KFVHXT0EKPRY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:42.289" grpc.time_ms=33.641 negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc2161vm-test-run-nixbot-gitlab> gitlab # [  312.505306] gitaly[1370]: time="2026-06-29T12:01:42.360Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3CC34F8J33QX6F9V2H5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T12:02:12.259" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:42.359" grpc.time_ms=1.116 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2162vm-test-run-nixbot-gitlab> gitlab # [  312.520163] nixbot[3805]: {"ts": "2026-06-29T12:01:42.364Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"}2163vm-test-run-nixbot-gitlab> gitlab # [  312.522310] nix-daemon[4102]: unexpected Nix daemon error: error: interrupted by the user2164vm-test-run-nixbot-gitlab> gitlab # [  312.551529] gitaly[1370]: time="2026-06-29T12:01:42.415Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3CC34F8J33QX6F9V2H5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:12.314" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:42.414" grpc.time_ms=0.876 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2165vm-test-run-nixbot-gitlab> gitlab # [  312.571299] gitaly[1370]: time="2026-06-29T12:01:42.435Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3EF15DE2D46QPT6AB758 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:12.234" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:42.434" grpc.time_ms=0.584 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2166vm-test-run-nixbot-gitlab> gitlab # [  312.590830] gitaly[1370]: time="2026-06-29T12:01:42.452Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=592 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3EF15DE2D46QPT6AB758 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-29T12:02:12.243" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:42.444" grpc.time_ms=8.568 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2167vm-test-run-nixbot-gitlab> gitlab # [  312.608756] gitaly[1370]: time="2026-06-29T12:01:42.465Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3EF15DE2D46QPT6AB758 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T12:01:52.088" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:42.458" grpc.time_ms=6.354 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2168vm-test-run-nixbot-gitlab> gitlab # [  312.624972] gitaly[1370]: time="2026-06-29T12:01:42.466Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3EG0KA31G0W5J89SE1VR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:12.465" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:42.465" grpc.time_ms=0.747 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2169vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=084809b87ce4cd362622ba41e849f7e129b03d15'2170vm-test-run-nixbot-gitlab> gitlab # [  312.639949] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:42.487Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"810e06a475f7b12342fdb3d8","created_at":"2026-06-29T12:01:40.706Z","trace_propagation_headers":{"sentry-trace":"51d35019551a421086939d99051becd1-6f77540e4680435b","baggage":"sentry-trace_id=51d35019551a421086939d99051becd1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3CC34F8J33QX6F9V2H5S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:40.715Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-810e06a475f7b12342fdb3d8: done: 0.426313 sec","job_status":"done","queue_duration_s":1.345305,"scheduling_latency_s":1.345347,"gitaly_calls":3,"gitaly_duration_s":0.024624,"redis_calls":48,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.015683,"redis_read_bytes":208,"redis_write_bytes":6803,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001179,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001502,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.0055,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.000307,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.003253,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000539,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.00019,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002034,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.058,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.327867,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.36129046,"duration_s":0.426313,"completed_at":"2026-06-29T12:01:42.486Z","load_balancing_strategy":"primary","db_duration_s":0.063572,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2171vm-test-run-nixbot-gitlab> gitlab # [  312.695009] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:42.488Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","ea74f6434df1abf6c443a28e"],"class":"WebHooks::LogExecutionWorker","jid":"c2ac8a2657bbee39397ec6a2","created_at":"2026-06-29T12:01:42.023Z","trace_propagation_headers":{"sentry-trace":"f40b0a3c6a1e4971b7234644bbdbda88-f5ee4c5e6a2a404e","baggage":"sentry-trace_id=f40b0a3c6a1e4971b7234644bbdbda88,sentry-release=Unknown"},"correlation_id":"01KW9M3BVWWCP2RZMADEK6NG0J","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dbf6fc727d6afbb24d68b6d2382b7e7f720ab2f9cb7d7bb3a19ffd1e00d29621","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:42.036Z","job_size_bytes":2507,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-c2ac8a2657bbee39397ec6a2: start","job_status":"start","queue_duration_s":0.451986,"scheduling_latency_s":0.452015}2172vm-test-run-nixbot-gitlab> gitlab # [  312.712860] gitaly[1370]: time="2026-06-29T12:01:42.470Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3CC34F8J33QX6F9V2H5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:12.369" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:42.469" grpc.time_ms=0.616 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2173vm-test-run-nixbot-gitlab> gitlab # [  312.726619] gitaly[1370]: time="2026-06-29T12:01:42.484Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3EF15DE2D46QPT6AB758 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T12:01:52.113" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:42.484" grpc.time_ms=0.269 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2174vm-test-run-nixbot-gitlab> gitlab # [  312.737849] gitaly[1370]: time="2026-06-29T12:01:42.494Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3EF15DE2D46QPT6AB758 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T12:01:52.117" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:42.487" grpc.time_ms=6.88 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2175vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=084809b87ce4cd362622ba41e849f7e129b03d15', in 0.08 seconds)2176vm-test-run-nixbot-gitlab> [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '084809b87ce4cd362622ba41e849f7e129b03d15', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-06-29T12:01:42.356427Z', 'started_at': None, 'finished_at': None}]2177vm-test-run-nixbot-gitlab> (finished: subtest: push delivers the webhook, in 3.52 seconds)2178vm-test-run-nixbot-gitlab> subtest: eval and build post commit statuses2179vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/084809b87ce4cd362622ba41e849f7e129b03d15/statuses2180vm-test-run-nixbot-gitlab> gitlab # [  312.751112] gitaly[1370]: time="2026-06-29T12:01:42.539Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M3EF15DE2D46QPT6AB758 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:42.538" grpc.time_ms=0.387 pid=1370 span.kind=server system=grpc2181vm-test-run-nixbot-gitlab> gitlab # [  312.763803] gitaly[1370]: time="2026-06-29T12:01:42.561Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M3EF15DE2D46QPT6AB758 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4157 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:42.560" grpc.time_ms=0.703 pid=1370 span.kind=server system=grpc2182vm-test-run-nixbot-gitlab> gitlab # [  312.773945] gitaly[1370]: time="2026-06-29T12:01:42.572Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3EG0KA31G0W5J89SE1VR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T12:01:52.141" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:42.561" grpc.time_ms=10.306 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2183vm-test-run-nixbot-gitlab> gitlab # [  312.788284] nixbot[3805]: INFO:     ::ffff:127.0.0.1:42886 - "GET /api/repos/gitlab/root/test-flake/builds?commit=084809b87ce4cd362622ba41e849f7e129b03d15 HTTP/1.1" 200 OK2184vm-test-run-nixbot-gitlab> gitlab # [  312.790420] gitaly[1370]: time="2026-06-29T12:01:42.576Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3EF15DE2D46QPT6AB758 diskcache=52a96788-4538-42f4-9837-9e2fb20d1385 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T12:01:52.126" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:42.496" pid=1370 ref=refs/keep-around/084809b87ce4cd362622ba41e849f7e129b03d15 remote_ip="2001:db8:1::1" revision=084809b87ce4cd362622ba41e849f7e129b03d15 span.kind=server system=grpc username=root2185vm-test-run-nixbot-gitlab> gitlab # [  312.802433] gitaly[1370]: time="2026-06-29T12:01:42.576Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=48 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5207 command.oublock=8 command.real_time_ms=73 command.spawn_token_fork_ms=2 command.system_time_ms=30 command.user_time_ms=18 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3EF15DE2D46QPT6AB758 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T12:01:52.126" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T12:01:42.496" grpc.time_ms=80.034 pid=1370 ref=refs/keep-around/084809b87ce4cd362622ba41e849f7e129b03d15 remote_ip="2001:db8:1::1" revision=084809b87ce4cd362622ba41e849f7e129b03d15 span.kind=server system=grpc username=root2186vm-test-run-nixbot-gitlab> gitlab # [  312.817888] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:42.661Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","ea74f6434df1abf6c443a28e"],"class":"WebHooks::LogExecutionWorker","jid":"c2ac8a2657bbee39397ec6a2","created_at":"2026-06-29T12:01:42.023Z","trace_propagation_headers":{"sentry-trace":"f40b0a3c6a1e4971b7234644bbdbda88-f5ee4c5e6a2a404e","baggage":"sentry-trace_id=f40b0a3c6a1e4971b7234644bbdbda88,sentry-release=Unknown"},"correlation_id":"01KW9M3BVWWCP2RZMADEK6NG0J","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dbf6fc727d6afbb24d68b6d2382b7e7f720ab2f9cb7d7bb3a19ffd1e00d29621","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:42.036Z","job_size_bytes":2507,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-c2ac8a2657bbee39397ec6a2: done: 0.173165 sec","job_status":"done","queue_duration_s":0.451986,"scheduling_latency_s":0.452015,"redis_calls":10,"redis_duration_s":0.003393,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000393,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002414,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000586,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.045,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.045,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.140537,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033356,"duration_s":0.173165,"completed_at":"2026-06-29T12:01:42.661Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017302,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2187vm-test-run-nixbot-gitlab> gitlab # [  312.863584] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:42.664Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"19347e08391941b56b8f0324","created_at":"2026-06-29T12:01:42.329Z","trace_propagation_headers":{"sentry-trace":"51d35019551a421086939d99051becd1-a74e1605d06e4eb1","baggage":"sentry-trace_id=51d35019551a421086939d99051becd1,sentry-release=Unknown"},"correlation_id":"01KW9M3CC34F8J33QX6F9V2H5S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:42.337Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-19347e08391941b56b8f0324: start","job_status":"start","queue_duration_s":0.326741,"scheduling_latency_s":0.326763}2188vm-test-run-nixbot-gitlab> gitlab # [  312.882947] gitaly[1370]: time="2026-06-29T12:01:42.582Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3EF15DE2D46QPT6AB758 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:12.380" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:42.581" grpc.time_ms=0.322 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2189vm-test-run-nixbot-gitlab> gitlab # [  312.896706] nixbot[3805]: {"ts": "2026-06-29T12:01:42.718Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/084809b87ce4cd362622ba41e849f7e129b03d15 \"HTTP/1.1 201 Created\""}2190vm-test-run-nixbot-gitlab> gitlab # [  312.899529] nixbot[3805]: {"ts": "2026-06-29T12:01:42.731Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-084809b87ce4-681d1e2a", "argv0": "bwrap"}2191vm-test-run-nixbot-gitlab> gitlab # [  312.902174] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:42 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/084809b87ce4cd362622ba41e849f7e129b03d15 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 3482192vm-test-run-nixbot-gitlab> gitlab # [  312.904815] gitaly[1370]: time="2026-06-29T12:01:42.759Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3ERCV32KBR4VFXRT71X5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:12.556" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:42.756" grpc.time_ms=2.421 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2193vm-test-run-nixbot-gitlab> gitlab # [  312.934752] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:42 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/084809b87ce4cd362622ba41e849f7e129b03d15/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1302194vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/084809b87ce4cd362622ba41e849f7e129b03d15/statuses, in 0.19 seconds)2195vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '084809b87ce4cd362622ba41e849f7e129b03d15', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T12:01:42.653Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2196vm-test-run-nixbot-gitlab> gitlab # [  312.985331] nix-daemon[4097]: accepted connection from pid 4191, user nixbot2197vm-test-run-nixbot-gitlab> gitlab # [  312.989708] nix-daemon[4097]: accepted connection from pid 4192, user nixbot2198vm-test-run-nixbot-gitlab> gitlab # [  313.014087] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:42 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/55d99a0d7974c5e56383b279aac3e27df907757a HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 4772199vm-test-run-nixbot-gitlab> gitlab # [  313.016918] nixbot[3805]: {"ts": "2026-06-29T12:01:42.880Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/55d99a0d7974c5e56383b279aac3e27df907757a \"HTTP/1.1 201 Created\""}2200vm-test-run-nixbot-gitlab> gitlab # [  313.059667] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:42.923Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"19347e08391941b56b8f0324","created_at":"2026-06-29T12:01:42.329Z","trace_propagation_headers":{"sentry-trace":"51d35019551a421086939d99051becd1-a74e1605d06e4eb1","baggage":"sentry-trace_id=51d35019551a421086939d99051becd1,sentry-release=Unknown"},"correlation_id":"01KW9M3CC34F8J33QX6F9V2H5S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:42.337Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-19347e08391941b56b8f0324: done: 0.258635 sec","job_status":"done","queue_duration_s":0.326741,"scheduling_latency_s":0.326763,"redis_calls":10,"redis_duration_s":0.003511,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000822,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002185,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000504,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.107,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.104,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004723,"duration_s":0.258635,"completed_at":"2026-06-29T12:01:42.922Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.10667,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2201vm-test-run-nixbot-gitlab> gitlab # [  313.119807] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:42.930Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"e581b5969391d90300edea2e","created_at":"2026-06-29T12:01:42.586Z","trace_propagation_headers":{"sentry-trace":"215e0afaa2d9411bbb397c126ececc00-dab99ffbce8c46f2","baggage":"sentry-trace_id=215e0afaa2d9411bbb397c126ececc00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3EF15DE2D46QPT6AB758","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:42.589Z","job_size_bytes":65,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-e581b5969391d90300edea2e: start","job_status":"start","queue_duration_s":0.340654,"scheduling_latency_s":0.34069}2202vm-test-run-nixbot-gitlab> gitlab # [  313.138505] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:42.979Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"e581b5969391d90300edea2e","created_at":"2026-06-29T12:01:42.586Z","trace_propagation_headers":{"sentry-trace":"215e0afaa2d9411bbb397c126ececc00-dab99ffbce8c46f2","baggage":"sentry-trace_id=215e0afaa2d9411bbb397c126ececc00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3EF15DE2D46QPT6AB758","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:42.589Z","job_size_bytes":65,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-e581b5969391d90300edea2e: done: 0.048937 sec","job_status":"done","queue_duration_s":0.340654,"scheduling_latency_s":0.34069,"redis_calls":7,"redis_duration_s":0.006582,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004629,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001953,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048937,"completed_at":"2026-06-29T12:01:42.979Z","load_balancing_strategy":"primary","db_duration_s":0.004356,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2203vm-test-run-nixbot-gitlab> gitlab # [  313.182386] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:42.982Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"47ccfe56062c1c09b696f738","created_at":"2026-06-29T12:01:42.590Z","trace_propagation_headers":{"sentry-trace":"215e0afaa2d9411bbb397c126ececc00-dab99ffbce8c46f2","baggage":"sentry-trace_id=215e0afaa2d9411bbb397c126ececc00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3EF15DE2D46QPT6AB758","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:42.593Z","job_size_bytes":65,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-47ccfe56062c1c09b696f738: start","job_status":"start","queue_duration_s":0.388261,"scheduling_latency_s":0.388293}2204vm-test-run-nixbot-gitlab> gitlab # [  313.200314] gitaly[1370]: time="2026-06-29T12:01:42.946Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3EZ5DCSXV7X9QBPT0B83 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:12.042" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:42.942" grpc.time_ms=3.02 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2205vm-test-run-nixbot-gitlab> gitlab # [  313.213278] nixbot[3805]: {"ts": "2026-06-29T12:01:43.062Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/55d99a0d7974c5e56383b279aac3e27df907757a \"HTTP/1.1 201 Created\""}2206vm-test-run-nixbot-gitlab> gitlab # [  313.216082] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:43 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/55d99a0d7974c5e56383b279aac3e27df907757a HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 1732207vm-test-run-nixbot-gitlab> gitlab # [  313.218647] gitaly[1370]: time="2026-06-29T12:01:42.990Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3EZ5DCSXV7X9QBPT0B83 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T12:01:52.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:42.979" grpc.time_ms=10.733 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2208vm-test-run-nixbot-gitlab> gitlab # [  313.232266] gitaly[1370]: time="2026-06-29T12:01:43.060Z" level=info msg="burden monitor stats" pid=1370 top_10_by_cpu="[]" top_10_by_memory="[]"2209vm-test-run-nixbot-gitlab> gitlab # [  313.290030] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:43.152Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"47ccfe56062c1c09b696f738","created_at":"2026-06-29T12:01:42.590Z","trace_propagation_headers":{"sentry-trace":"215e0afaa2d9411bbb397c126ececc00-dab99ffbce8c46f2","baggage":"sentry-trace_id=215e0afaa2d9411bbb397c126ececc00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3EF15DE2D46QPT6AB758","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:42.593Z","job_size_bytes":65,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-47ccfe56062c1c09b696f738: done: 0.169937 sec","job_status":"done","queue_duration_s":0.388261,"scheduling_latency_s":0.388293,"redis_calls":10,"redis_duration_s":0.003346,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001575,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001771,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.134935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.169937,"completed_at":"2026-06-29T12:01:43.152Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024868,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2210vm-test-run-nixbot-gitlab> gitlab # [  313.353150] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:43.157Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"8277997edcb4e17a2a641c3a","created_at":"2026-06-29T12:01:42.660Z","trace_propagation_headers":{"sentry-trace":"215e0afaa2d9411bbb397c126ececc00-dab99ffbce8c46f2","baggage":"sentry-trace_id=215e0afaa2d9411bbb397c126ececc00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3EF15DE2D46QPT6AB758","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:42.663Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8277997edcb4e17a2a641c3a: start","job_status":"start","queue_duration_s":0.493397,"scheduling_latency_s":0.493419}2211vm-test-run-nixbot-gitlab> gitlab # [  313.475061] gitaly[1370]: time="2026-06-29T12:01:43.339Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3EF15DE2D46QPT6AB758 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T12:02:13.236" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:43.337" grpc.time_ms=1.578 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2212vm-test-run-nixbot-gitlab> gitlab # [  313.556124] gitaly[1370]: time="2026-06-29T12:01:43.420Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3EF15DE2D46QPT6AB758 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:13.318" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:43.418" grpc.time_ms=1.216 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2213vm-test-run-nixbot-gitlab> gitlab # [  313.622990] gitaly[1370]: time="2026-06-29T12:01:43.487Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3EF15DE2D46QPT6AB758 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:13.384" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:43.485" grpc.time_ms=1.312 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2214vm-test-run-nixbot-gitlab> gitlab # [  313.660185] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:43.510Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"8277997edcb4e17a2a641c3a","created_at":"2026-06-29T12:01:42.660Z","trace_propagation_headers":{"sentry-trace":"215e0afaa2d9411bbb397c126ececc00-dab99ffbce8c46f2","baggage":"sentry-trace_id=215e0afaa2d9411bbb397c126ececc00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3EF15DE2D46QPT6AB758","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:42.663Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8277997edcb4e17a2a641c3a: done: 0.352857 sec","job_status":"done","queue_duration_s":0.493397,"scheduling_latency_s":0.493419,"gitaly_calls":3,"gitaly_duration_s":0.017989,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.012687,"redis_read_bytes":98,"redis_write_bytes":6014,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001353,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003961,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000344,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001913,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000361,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.00208,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001322,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.055,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.021,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.281894,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.29885162,"duration_s":0.352857,"completed_at":"2026-06-29T12:01:43.509Z","load_balancing_strategy":"primary","db_duration_s":0.057315,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2215vm-test-run-nixbot-gitlab> gitlab # [  313.737214] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:43.515Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"384de31aa341d9b584ebc118","created_at":"2026-06-29T12:01:42.854Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-97a7d696a98a4ea5","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:42.859Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-384de31aa341d9b584ebc118: start","job_status":"start","queue_duration_s":0.656591,"scheduling_latency_s":0.656616}2216vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/084809b87ce4cd362622ba41e849f7e129b03d15/statuses2217vm-test-run-nixbot-gitlab> gitlab # [  314.025213] gitaly[1370]: time="2026-06-29T12:01:43.887Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M3EG0KA31G0W5J89SE1VR grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:43.887" grpc.time_ms=0.564 pid=1370 span.kind=server system=grpc2218vm-test-run-nixbot-gitlab> gitlab # [  314.058907] gitaly[1370]: time="2026-06-29T12:01:43.923Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M3EG0KA31G0W5J89SE1VR grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:43.922" grpc.time_ms=0.683 pid=1370 span.kind=server system=grpc2219vm-test-run-nixbot-gitlab> gitlab # [  314.091571] gitaly[1370]: time="2026-06-29T12:01:43.955Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M3EG0KA31G0W5J89SE1VR grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:43.955" grpc.time_ms=0.524 pid=1370 span.kind=server system=grpc2220vm-test-run-nixbot-gitlab> gitlab # [  314.115326] gitaly[1370]: time="2026-06-29T12:01:43.966Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3EG0KA31G0W5J89SE1VR diskcache=f198acca-7a0b-43ef-8aa8-e7df83286441 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T12:02:13.737" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:43.838" pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2221vm-test-run-nixbot-gitlab> gitlab # [  314.134935] gitaly[1370]: time="2026-06-29T12:01:43.967Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=124 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=6916 command.oublock=0 command.real_time_ms=120 command.spawn_token_fork_ms=1 command.system_time_ms=93 command.user_time_ms=31 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3EG0KA31G0W5J89SE1VR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T12:02:13.737" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:43.838" grpc.time_ms=128.489 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2222vm-test-run-nixbot-gitlab> gitlab # [  314.155506] gitaly[1370]: time="2026-06-29T12:01:43.976Z" level=info msg="finished unary call with code OK" catfile.duration_ms=4 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=4 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3FXXBGPZNX6QW99HJB8J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:13.071" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:43.971" grpc.time_ms=4.815 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2223vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/084809b87ce4cd362622ba41e849f7e129b03d15/statuses, in 0.23 seconds)2224vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '084809b87ce4cd362622ba41e849f7e129b03d15', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T12:01:42.653Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2225vm-test-run-nixbot-gitlab> gitlab # [  314.174791] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:44 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/084809b87ce4cd362622ba41e849f7e129b03d15/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1652226vm-test-run-nixbot-gitlab> gitlab # [  314.329674] gitaly[1370]: time="2026-06-29T12:01:44.193Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3EG0KA31G0W5J89SE1VR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T12:02:14.091" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:44.192" grpc.time_ms=1.39 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2227vm-test-run-nixbot-gitlab> gitlab # [  314.453008] gitaly[1370]: time="2026-06-29T12:01:44.317Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3EG0KA31G0W5J89SE1VR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:14.214" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:44.315" grpc.time_ms=1.296 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2228vm-test-run-nixbot-gitlab> gitlab # [  314.586447] gitaly[1370]: time="2026-06-29T12:01:44.450Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3EG0KA31G0W5J89SE1VR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:14.348" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:44.449" grpc.time_ms=0.994 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2229vm-test-run-nixbot-gitlab> gitlab # [  314.624421] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:44.483Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"384de31aa341d9b584ebc118","created_at":"2026-06-29T12:01:42.854Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-97a7d696a98a4ea5","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:42.859Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2415,"job_status":"rescheduled","message":"PipelineProcessWorker JID-384de31aa341d9b584ebc118: rescheduled"}2230vm-test-run-nixbot-gitlab> gitlab # [  314.670221] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:44.526Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"384de31aa341d9b584ebc118","created_at":"2026-06-29T12:01:42.854Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-97a7d696a98a4ea5","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:42.859Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-384de31aa341d9b584ebc118: done: 1.010946 sec","job_status":"done","queue_duration_s":0.656591,"scheduling_latency_s":0.656616,"gitaly_calls":4,"gitaly_duration_s":0.154553,"redis_calls":101,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.071103,"redis_read_bytes":185,"redis_write_bytes":23554,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.004682,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.008851,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.015986,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.00462,"redis_queues_read_bytes":25,"redis_queues_write_bytes":12386,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.020608,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.006485,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000715,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004474,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.087,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.049,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.053,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.054,"db_ci_duration_s":0.033,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.742635,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.9043144,"duration_s":1.010946,"completed_at":"2026-06-29T12:01:44.526Z","load_balancing_strategy":"primary","db_duration_s":0.093298,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2231vm-test-run-nixbot-gitlab> gitlab # [  314.746805] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:44.539Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"a92ef43359acd34b5897d675","created_at":"2026-06-29T12:01:43.308Z","trace_propagation_headers":{"sentry-trace":"215e0afaa2d9411bbb397c126ececc00-5376f890e66c4278","baggage":"sentry-trace_id=215e0afaa2d9411bbb397c126ececc00,sentry-release=Unknown"},"correlation_id":"01KW9M3EF15DE2D46QPT6AB758","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:43.314Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-a92ef43359acd34b5897d675: start","job_status":"start","queue_duration_s":1.224895,"scheduling_latency_s":1.224933}2232vm-test-run-nixbot-gitlab> gitlab # [  314.776683] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:44.636Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"a92ef43359acd34b5897d675","created_at":"2026-06-29T12:01:43.308Z","trace_propagation_headers":{"sentry-trace":"215e0afaa2d9411bbb397c126ececc00-5376f890e66c4278","baggage":"sentry-trace_id=215e0afaa2d9411bbb397c126ececc00,sentry-release=Unknown"},"correlation_id":"01KW9M3EF15DE2D46QPT6AB758","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:43.314Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-a92ef43359acd34b5897d675: done: 0.096954 sec","job_status":"done","queue_duration_s":1.224895,"scheduling_latency_s":1.224933,"redis_calls":7,"redis_duration_s":0.0027610000000000004,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002063,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000698,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.076201,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.096954,"completed_at":"2026-06-29T12:01:44.636Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008173,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2233vm-test-run-nixbot-gitlab> gitlab # [  314.831120] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:44.642Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"c8f0940198eb7a3e271c738e","created_at":"2026-06-29T12:01:28.469Z","trace_propagation_headers":{"sentry-trace":"b513e5d1e06f4a0c93182ad5c5ea3ece-4e1327d5784d43ec","baggage":"sentry-trace_id=b513e5d1e06f4a0c93182ad5c5ea3ece,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M30N1K7JXAJQT39A6XXQQ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:9f88f85afada9c142331ff7af8ab78c1f672ec8ef9d4ee60fdbf95bef450e365","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:43.992Z","retry_count":0,"job_size_bytes":451,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-c8f0940198eb7a3e271c738e: start","job_status":"start","queue_duration_s":0.65029,"scheduling_latency_s":0.650318}2234vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/084809b87ce4cd362622ba41e849f7e129b03d15/statuses2235vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/084809b87ce4cd362622ba41e849f7e129b03d15/statuses, in 0.16 seconds)2236vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '084809b87ce4cd362622ba41e849f7e129b03d15', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T12:01:42.653Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2237vm-test-run-nixbot-gitlab> gitlab # [  314.881353] sidekiq-cluster[2415]: {"severity":"WARN","time":"2026-06-29T12:01:44.740Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"c8f0940198eb7a3e271c738e","created_at":"2026-06-29T12:01:28.469Z","trace_propagation_headers":{"sentry-trace":"b513e5d1e06f4a0c93182ad5c5ea3ece-4e1327d5784d43ec","baggage":"sentry-trace_id=b513e5d1e06f4a0c93182ad5c5ea3ece,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M30N1K7JXAJQT39A6XXQQ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:9f88f85afada9c142331ff7af8ab78c1f672ec8ef9d4ee60fdbf95bef450e365","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:43.992Z","retry_count":0,"job_size_bytes":451,"pid":2415,"sidekiq_tid":"m2kv","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-c8f0940198eb7a3e271c738e: fail: 0.088255 sec","job_status":"fail","queue_duration_s":0.65029,"scheduling_latency_s":0.650318,"redis_calls":7,"redis_duration_s":0.0024720000000000002,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001796,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000676,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.088255,"completed_at":"2026-06-29T12:01:44.730Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/w9mp1gyfs2qdv12r0ji70idzndmxwz2s-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/p8ylyr0sj9sxcknqcpja13nypqixk2dp-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/dndr7b4a17339dsm67vzb6nv4p2kp82q-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/zx6ndpscxs64hnidz69ab430ccms6rak-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.008182,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2238vm-test-run-nixbot-gitlab> gitlab # [  315.387073] gitaly[1370]: time="2026-06-29T12:01:45.163Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3H4CP1MQ82QTMKDPMWG7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:15.162" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:45.162" grpc.time_ms=0.779 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2239vm-test-run-nixbot-gitlab> gitlab # [  315.401255] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:45 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/084809b87ce4cd362622ba41e849f7e129b03d15/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 922240vm-test-run-nixbot-gitlab> gitlab # [  315.403860] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:44.752Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8e36bc8648aeaa0cf33c63d0","created_at":"2026-06-29T12:01:43.782Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:43.798Z","job_size_bytes":23,"pid":2415,"sidekiq_tid":"m7lv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8e36bc8648aeaa0cf33c63d0: start","job_status":"start","queue_duration_s":0.954148,"scheduling_latency_s":0.954183}2241vm-test-run-nixbot-gitlab> gitlab # [  315.796430] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:45.658Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8e36bc8648aeaa0cf33c63d0","created_at":"2026-06-29T12:01:43.782Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:43.798Z","job_size_bytes":23,"pid":2415,"sidekiq_tid":"m7lv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8e36bc8648aeaa0cf33c63d0: done: 0.907558 sec","job_status":"done","queue_duration_s":0.954148,"scheduling_latency_s":0.954183,"redis_calls":15,"redis_duration_s":0.006478,"redis_read_bytes":13,"redis_write_bytes":3012,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000795,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.000489,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1627,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002699,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.00178,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000715,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.01,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.01,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.329318,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00269718,"duration_s":0.907558,"completed_at":"2026-06-29T12:01:45.658Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.029773,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2242vm-test-run-nixbot-gitlab> gitlab # [  315.910913] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:45.661Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"5257e0720e9c9da1e13ed73d","created_at":"2026-06-29T12:01:45.095Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-3d5b6edf81c74e6f","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:6cd166958644dd29ef2c0f9116efa43db1ca00fedd85be2113a84de4797f61e7","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:45.105Z","job_size_bytes":464,"pid":2415,"sidekiq_tid":"m7lv","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-5257e0720e9c9da1e13ed73d: start","job_status":"start","queue_duration_s":0.556273,"scheduling_latency_s":0.556298}2243vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/084809b87ce4cd362622ba41e849f7e129b03d15/statuses2244vm-test-run-nixbot-gitlab> gitlab # [  315.958048] sidekiq-cluster[2415]: {"severity":"WARN","time":"2026-06-29T12:01:45.717Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"5257e0720e9c9da1e13ed73d","created_at":"2026-06-29T12:01:45.095Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-3d5b6edf81c74e6f","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:6cd166958644dd29ef2c0f9116efa43db1ca00fedd85be2113a84de4797f61e7","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:45.105Z","job_size_bytes":464,"pid":2415,"sidekiq_tid":"m7lv","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-5257e0720e9c9da1e13ed73d: fail: 0.054832 sec","job_status":"fail","queue_duration_s":0.556273,"scheduling_latency_s":0.556298,"redis_calls":7,"redis_duration_s":0.002016,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001439,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000577,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.054832,"completed_at":"2026-06-29T12:01:45.716Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/w9mp1gyfs2qdv12r0ji70idzndmxwz2s-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/p8ylyr0sj9sxcknqcpja13nypqixk2dp-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/dndr7b4a17339dsm67vzb6nv4p2kp82q-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/zx6ndpscxs64hnidz69ab430ccms6rak-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.014554,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2245vm-test-run-nixbot-gitlab> gitlab # [  316.425170] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:45.721Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a14bdb5f0f4533fdf3a92c8e","created_at":"2026-06-29T12:01:43.983Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.016Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a14bdb5f0f4533fdf3a92c8e: start","job_status":"start","queue_duration_s":1.705087,"scheduling_latency_s":1.705136}2246vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/084809b87ce4cd362622ba41e849f7e129b03d15/statuses, in 0.15 seconds)2247vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '084809b87ce4cd362622ba41e849f7e129b03d15', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T12:01:42.653Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2248vm-test-run-nixbot-gitlab> gitlab # [  316.442954] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:45.781Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a14bdb5f0f4533fdf3a92c8e","created_at":"2026-06-29T12:01:43.983Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.016Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a14bdb5f0f4533fdf3a92c8e: done: 0.059792 sec","job_status":"done","queue_duration_s":1.705087,"scheduling_latency_s":1.705136,"redis_calls":10,"redis_duration_s":0.004077,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000752,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002487,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000838,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00064122,"duration_s":0.059792,"completed_at":"2026-06-29T12:01:45.781Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007771,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2249vm-test-run-nixbot-gitlab> gitlab # [  316.487148] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:45.784Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"993767e0c6e4d2380c47f9e1","created_at":"2026-06-29T12:01:44.064Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.082Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-993767e0c6e4d2380c47f9e1: start","job_status":"start","queue_duration_s":1.702033,"scheduling_latency_s":1.702063}2250vm-test-run-nixbot-gitlab> gitlab # [  316.503312] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:45.841Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"993767e0c6e4d2380c47f9e1","created_at":"2026-06-29T12:01:44.064Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.082Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-993767e0c6e4d2380c47f9e1: done: 0.056733 sec","job_status":"done","queue_duration_s":1.702033,"scheduling_latency_s":1.702063,"redis_calls":10,"redis_duration_s":0.003498,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000538,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002335,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000625,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031974,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00058916,"duration_s":0.056733,"completed_at":"2026-06-29T12:01:45.841Z","load_balancing_strategy":"primary","db_duration_s":0.014643,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2251vm-test-run-nixbot-gitlab> gitlab # [  316.547874] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:45.843Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"2c6662b25d968d7b356850d6","created_at":"2026-06-29T12:01:44.107Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.117Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-2c6662b25d968d7b356850d6: start","job_status":"start","queue_duration_s":1.72611,"scheduling_latency_s":1.726135}2252vm-test-run-nixbot-gitlab> gitlab # [  316.563711] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:45.890Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"2c6662b25d968d7b356850d6","created_at":"2026-06-29T12:01:44.107Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.117Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-2c6662b25d968d7b356850d6: done: 0.047036 sec","job_status":"done","queue_duration_s":1.72611,"scheduling_latency_s":1.726135,"redis_calls":13,"redis_duration_s":0.003856,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000635,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001711,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000929,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000581,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039294,"duration_s":0.047036,"completed_at":"2026-06-29T12:01:45.890Z","load_balancing_strategy":"primary","db_duration_s":0.007426,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2253vm-test-run-nixbot-gitlab> gitlab # [  316.609570] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:45.893Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"53e02c9d5c97c4db803f4f4f","created_at":"2026-06-29T12:01:44.118Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.132Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-53e02c9d5c97c4db803f4f4f: start","job_status":"start","queue_duration_s":1.76118,"scheduling_latency_s":1.761202}2254vm-test-run-nixbot-gitlab> gitlab # [  316.626168] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:45.947Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"53e02c9d5c97c4db803f4f4f","created_at":"2026-06-29T12:01:44.118Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.132Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-53e02c9d5c97c4db803f4f4f: done: 0.054415 sec","job_status":"done","queue_duration_s":1.76118,"scheduling_latency_s":1.761202,"redis_calls":10,"redis_duration_s":0.004445,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000556,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002755,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001134,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00116876,"duration_s":0.054415,"completed_at":"2026-06-29T12:01:45.947Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00566,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2255vm-test-run-nixbot-gitlab> gitlab # [  316.672862] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:45.951Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ee46391857f0ab809fcbe7df","created_at":"2026-06-29T12:01:44.133Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.145Z","job_size_bytes":19,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ee46391857f0ab809fcbe7df: start","job_status":"start","queue_duration_s":1.805692,"scheduling_latency_s":1.805737}2256vm-test-run-nixbot-gitlab> gitlab # [  316.720790] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:46.018Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ee46391857f0ab809fcbe7df","created_at":"2026-06-29T12:01:44.133Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.145Z","job_size_bytes":19,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ee46391857f0ab809fcbe7df: done: 0.067513 sec","job_status":"done","queue_duration_s":1.805692,"scheduling_latency_s":1.805737,"redis_calls":11,"redis_duration_s":0.004039,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000669,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002689,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000681,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043487,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00054616,"duration_s":0.067513,"completed_at":"2026-06-29T12:01:46.018Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008191,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2257vm-test-run-nixbot-gitlab> gitlab # [  316.794861] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:46.020Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"668b90015a0110a8752a154f","created_at":"2026-06-29T12:01:44.160Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.172Z","job_size_bytes":65,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-668b90015a0110a8752a154f: start","job_status":"start","queue_duration_s":1.84857,"scheduling_latency_s":1.848602}2258vm-test-run-nixbot-gitlab> gitlab # [  316.819384] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:46.066Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"668b90015a0110a8752a154f","created_at":"2026-06-29T12:01:44.160Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.172Z","job_size_bytes":65,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-668b90015a0110a8752a154f: done: 0.045702 sec","job_status":"done","queue_duration_s":1.84857,"scheduling_latency_s":1.848602,"redis_calls":10,"redis_duration_s":0.0029909999999999997,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000563,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001774,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000654,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005237,"duration_s":0.045702,"completed_at":"2026-06-29T12:01:46.066Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006757,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2259vm-test-run-nixbot-gitlab> gitlab # [  316.870365] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:46.069Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"368b4393c9edc114fb543da3","created_at":"2026-06-29T12:01:44.173Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.179Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-368b4393c9edc114fb543da3: start","job_status":"start","queue_duration_s":1.889352,"scheduling_latency_s":1.889379}2260vm-test-run-nixbot-gitlab> gitlab # [  316.917052] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:46.114Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"368b4393c9edc114fb543da3","created_at":"2026-06-29T12:01:44.173Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.179Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-368b4393c9edc114fb543da3: done: 0.045783 sec","job_status":"done","queue_duration_s":1.889352,"scheduling_latency_s":1.889379,"redis_calls":7,"redis_duration_s":0.002264,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001632,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000632,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031847,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.045783,"completed_at":"2026-06-29T12:01:46.114Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006936,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2261vm-test-run-nixbot-gitlab> gitlab # [  316.984661] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:46.117Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"fc4bf8be2d4f63d713933090","created_at":"2026-06-29T12:01:44.322Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.340Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-fc4bf8be2d4f63d713933090: start","job_status":"start","queue_duration_s":1.776953,"scheduling_latency_s":1.776978}2262vm-test-run-nixbot-gitlab> gitlab # [  317.002493] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:46.169Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"fc4bf8be2d4f63d713933090","created_at":"2026-06-29T12:01:44.322Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.340Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-fc4bf8be2d4f63d713933090: done: 0.052098 sec","job_status":"done","queue_duration_s":1.776953,"scheduling_latency_s":1.776978,"redis_calls":10,"redis_duration_s":0.002972,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000513,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001854,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000605,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00058358,"duration_s":0.052098,"completed_at":"2026-06-29T12:01:46.169Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015954,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2263vm-test-run-nixbot-gitlab> gitlab # [  317.046575] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:46.171Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"2a220c9c92bbfeffe1d85be3","created_at":"2026-06-29T12:01:44.341Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.354Z","job_size_bytes":6,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-2a220c9c92bbfeffe1d85be3: start","job_status":"start","queue_duration_s":1.816392,"scheduling_latency_s":1.816415}2264vm-test-run-nixbot-gitlab> gitlab # [  317.062700] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:46.202Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"2a220c9c92bbfeffe1d85be3","created_at":"2026-06-29T12:01:44.341Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.354Z","job_size_bytes":6,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-2a220c9c92bbfeffe1d85be3: done: 0.031592 sec","job_status":"done","queue_duration_s":1.816392,"scheduling_latency_s":1.816415,"redis_calls":10,"redis_duration_s":0.002859,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000522,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001712,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000625,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017941,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00046188,"duration_s":0.031592,"completed_at":"2026-06-29T12:01:46.202Z","load_balancing_strategy":"primary","db_duration_s":0.003994,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2265vm-test-run-nixbot-gitlab> gitlab # [  317.107158] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:46.204Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"16c381c07aa1563f935d70de","created_at":"2026-06-29T12:01:44.488Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.504Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-16c381c07aa1563f935d70de: start","job_status":"start","queue_duration_s":1.69945,"scheduling_latency_s":1.699471}2266vm-test-run-nixbot-gitlab> gitlab # [  317.123432] gitaly[1370]: time="2026-06-29T12:01:46.296Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3J801P239RPDTXVA192D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:16.094" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:46.295" grpc.time_ms=0.951 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2267vm-test-run-nixbot-gitlab> gitlab # [  317.136532] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/084809b87ce4cd362622ba41e849f7e129b03d15/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 852268vm-test-run-nixbot-gitlab> gitlab # [  317.139166] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:46.244Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"16c381c07aa1563f935d70de","created_at":"2026-06-29T12:01:44.488Z","trace_propagation_headers":{"sentry-trace":"39e8c99cb17c498fbf0c54270b47f3dc-b47aab6a49aa4fc4","baggage":"sentry-trace_id=39e8c99cb17c498fbf0c54270b47f3dc,sentry-release=Unknown"},"correlation_id":"01KW9M3EG0KA31G0W5J89SE1VR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:44.504Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-16c381c07aa1563f935d70de: done: 0.039698 sec","job_status":"done","queue_duration_s":1.69945,"scheduling_latency_s":1.699471,"redis_calls":8,"redis_duration_s":0.0036710000000000002,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002912,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000759,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039698,"completed_at":"2026-06-29T12:01:46.244Z","load_balancing_strategy":"primary","db_duration_s":0.005196,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2269vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/084809b87ce4cd362622ba41e849f7e129b03d15/statuses2270vm-test-run-nixbot-gitlab> gitlab # [  317.494404] gitaly[1370]: time="2026-06-29T12:01:47.358Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3K7DVSWQ3VGN7QANCY86 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:17.155" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:47.356" grpc.time_ms=1.187 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2271vm-test-run-nixbot-gitlab> gitlab # [  317.583920] gitaly[1370]: time="2026-06-29T12:01:47.448Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3K7DVSWQ3VGN7QANCY86 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T12:01:57.059" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:47.429" grpc.time_ms=17.81 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2272vm-test-run-nixbot-gitlab> gitlab # [  317.654931] gitaly[1370]: time="2026-06-29T12:01:47.518Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3KCTX1QNAYHM03DSZJTN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:17.517" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:47.517" grpc.time_ms=1.056 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2273vm-test-run-nixbot-gitlab> gitlab # [  317.696479] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:47.519Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"56ec7e5062c514f0644a5a30","created_at":"2026-06-29T12:01:47.511Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-01093cc8fc3d4168","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.517Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-56ec7e5062c514f0644a5a30: start","job_status":"start","queue_duration_s":0.00223,"scheduling_latency_s":0.002298}2274vm-test-run-nixbot-gitlab> gitlab # [  317.722483] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:47 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/084809b87ce4cd362622ba41e849f7e129b03d15 HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 3172275vm-test-run-nixbot-gitlab> gitlab # [  317.726802] nixbot[3805]: {"ts": "2026-06-29T12:01:47.565Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/084809b87ce4cd362622ba41e849f7e129b03d15 \"HTTP/1.1 201 Created\""}2276vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/084809b87ce4cd362622ba41e849f7e129b03d15/statuses, in 0.25 seconds)2277vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '084809b87ce4cd362622ba41e849f7e129b03d15', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-29T12:01:42.653Z', 'started_at': None, 'finished_at': '2026-06-29T12:01:47.491Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2278vm-test-run-nixbot-gitlab> gitlab # [  317.731704] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:47 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/084809b87ce4cd362622ba41e849f7e129b03d15/statuses HTTP/1.1" 200 671 "" "curl/8.20.0" 1692279vm-test-run-nixbot-gitlab> gitlab # [  317.779430] gitaly[1370]: time="2026-06-29T12:01:47.643Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3KHRM0A9TN7SX927MY7M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:17.441" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:47.642" grpc.time_ms=1.033 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2280vm-test-run-nixbot-gitlab> gitlab # [  317.841669] gitaly[1370]: time="2026-06-29T12:01:47.705Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3KHRM0A9TN7SX927MY7M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T12:01:57.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:47.688" grpc.time_ms=17.351 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2281vm-test-run-nixbot-gitlab> gitlab # [  317.874877] gitaly[1370]: time="2026-06-29T12:01:47.707Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M3K7DVSWQ3VGN7QANCY86 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:47.706" grpc.time_ms=1.097 pid=1370 span.kind=server system=grpc2282vm-test-run-nixbot-gitlab> gitlab # [  317.893688] gitaly[1370]: time="2026-06-29T12:01:47.747Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M3K7DVSWQ3VGN7QANCY86 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:47.746" grpc.time_ms=0.752 pid=1370 span.kind=server system=grpc2283vm-test-run-nixbot-gitlab> gitlab # [  317.911533] gitaly[1370]: time="2026-06-29T12:01:47.773Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9M3K7DVSWQ3VGN7QANCY86 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T12:01:47.772" grpc.time_ms=0.971 pid=1370 span.kind=server system=grpc2284vm-test-run-nixbot-gitlab> gitlab # [  317.927893] gitaly[1370]: time="2026-06-29T12:01:47.782Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3K7DVSWQ3VGN7QANCY86 diskcache=a8c77e8e-d06b-4d5b-9c9c-c56f7a81209a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T12:02:17.569" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:47.670" pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2285vm-test-run-nixbot-gitlab> gitlab # [  317.943525] gitaly[1370]: time="2026-06-29T12:01:47.782Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=118 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7505 command.oublock=0 command.real_time_ms=107 command.spawn_token_fork_ms=1 command.system_time_ms=64 command.user_time_ms=54 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3K7DVSWQ3VGN7QANCY86 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T12:02:17.569" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:47.670" grpc.time_ms=112.606 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2286vm-test-run-nixbot-gitlab> gitlab # [  317.962718] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:47 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/084809b87ce4cd362622ba41e849f7e129b03d15 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2132287vm-test-run-nixbot-gitlab> gitlab # [  317.966112] nixbot[3805]: {"ts": "2026-06-29T12:01:47.792Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/084809b87ce4cd362622ba41e849f7e129b03d15 \"HTTP/1.1 201 Created\""}2288vm-test-run-nixbot-gitlab> gitlab # [  317.988115] gitaly[1370]: time="2026-06-29T12:01:47.851Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3K7DVSWQ3VGN7QANCY86 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T12:02:17.749" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:47.850" grpc.time_ms=1.12 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2289vm-test-run-nixbot-gitlab> gitlab # [  318.006613] gitaly[1370]: time="2026-06-29T12:01:47.870Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3KRWE1TEKXJDF28BYZ89 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:17.868" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:47.869" grpc.time_ms=0.979 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2290vm-test-run-nixbot-gitlab> gitlab # [  318.040882] gitaly[1370]: time="2026-06-29T12:01:47.890Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3K7DVSWQ3VGN7QANCY86 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:17.789" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:47.889" grpc.time_ms=0.616 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2291vm-test-run-nixbot-gitlab> gitlab # [  318.065219] gitaly[1370]: time="2026-06-29T12:01:47.918Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3KRWE1TEKXJDF28BYZ89 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T12:01:57.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T12:01:47.908" grpc.time_ms=9.61 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2292vm-test-run-nixbot-gitlab> gitlab # [  318.088722] gitaly[1370]: time="2026-06-29T12:01:47.938Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3K7DVSWQ3VGN7QANCY86 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T12:02:17.837" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T12:01:47.938" grpc.time_ms=0.621 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2293vm-test-run-nixbot-gitlab> gitlab # [  318.108511] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:47.948Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"a62b7939f5a74a926664629c","created_at":"2026-06-29T12:01:47.946Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"56ec7e5062c514f0644a5a30","job_size_bytes":3,"pid":2415,"job_status":"deduplicated","message":"PipelineProcessWorker JID-a62b7939f5a74a926664629c: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2294vm-test-run-nixbot-gitlab> gitlab # [  318.132342] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:47.950Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"56ec7e5062c514f0644a5a30","created_at":"2026-06-29T12:01:47.511Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-01093cc8fc3d4168","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.517Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2415,"job_status":"rescheduled","message":"PipelineProcessWorker JID-56ec7e5062c514f0644a5a30: rescheduled"}2295vm-test-run-nixbot-gitlab> gitlab # [  318.152472] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:47.957Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"56ec7e5062c514f0644a5a30","created_at":"2026-06-29T12:01:47.511Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-01093cc8fc3d4168","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.517Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-56ec7e5062c514f0644a5a30: done: 0.438095 sec","job_status":"done","queue_duration_s":0.00223,"scheduling_latency_s":0.002298,"gitaly_calls":4,"gitaly_duration_s":0.124617,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.031316,"redis_read_bytes":262,"redis_write_bytes":23382,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.003531,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002934,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000772,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.008266,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14463,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.006755,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000468,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000651,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.004408,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.012,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.017,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.035,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.245566,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.3914482,"duration_s":0.438095,"completed_at":"2026-06-29T12:01:47.957Z","load_balancing_strategy":"primary","db_duration_s":0.058012,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2296vm-test-run-nixbot-gitlab> gitlab # [  318.211597] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:47.959Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"9346268b844d25adf978a488","created_at":"2026-06-29T12:01:47.655Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.659Z","job_size_bytes":23,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-9346268b844d25adf978a488: start","job_status":"start","queue_duration_s":0.299397,"scheduling_latency_s":0.299421}2297vm-test-run-nixbot-gitlab> gitlab # [  318.228493] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:47.996Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"9346268b844d25adf978a488","created_at":"2026-06-29T12:01:47.655Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.659Z","job_size_bytes":23,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-9346268b844d25adf978a488: done: 0.037239 sec","job_status":"done","queue_duration_s":0.299397,"scheduling_latency_s":0.299421,"redis_calls":7,"redis_duration_s":0.0012460000000000001,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000353,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037239,"completed_at":"2026-06-29T12:01:47.996Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006755,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2298vm-test-run-nixbot-gitlab> gitlab # [  318.284511] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:47.998Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"6ebeb3e99451cb2a309a32d5","created_at":"2026-06-29T12:01:47.789Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.792Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-6ebeb3e99451cb2a309a32d5: start","job_status":"start","queue_duration_s":0.205199,"scheduling_latency_s":0.205218}2299vm-test-run-nixbot-gitlab> gitlab # [  318.301537] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.021Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"6ebeb3e99451cb2a309a32d5","created_at":"2026-06-29T12:01:47.789Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.792Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-6ebeb3e99451cb2a309a32d5: done: 0.023142 sec","job_status":"done","queue_duration_s":0.205199,"scheduling_latency_s":0.205218,"redis_calls":7,"redis_duration_s":0.001727,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001325,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000402,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023142,"completed_at":"2026-06-29T12:01:48.021Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002697,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2300vm-test-run-nixbot-gitlab> gitlab # [  318.346085] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.022Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"166e7003097935757e738953","created_at":"2026-06-29T12:01:47.806Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.809Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-166e7003097935757e738953: start","job_status":"start","queue_duration_s":0.2131,"scheduling_latency_s":0.213118}2301vm-test-run-nixbot-gitlab> gitlab # [  318.362513] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.049Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"166e7003097935757e738953","created_at":"2026-06-29T12:01:47.806Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.809Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-166e7003097935757e738953: done: 0.027341 sec","job_status":"done","queue_duration_s":0.2131,"scheduling_latency_s":0.213118,"redis_calls":7,"redis_duration_s":0.001595,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001155,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00044,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027341,"completed_at":"2026-06-29T12:01:48.049Z","load_balancing_strategy":"primary","db_duration_s":0.005543,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2302vm-test-run-nixbot-gitlab> gitlab # [  318.405566] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.051Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"e8ac8faeb554683ce50d8a56","created_at":"2026-06-29T12:01:47.822Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.825Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-e8ac8faeb554683ce50d8a56: start","job_status":"start","queue_duration_s":0.226023,"scheduling_latency_s":0.226042}2303vm-test-run-nixbot-gitlab> gitlab # [  318.421615] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.078Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"e8ac8faeb554683ce50d8a56","created_at":"2026-06-29T12:01:47.822Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.825Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-e8ac8faeb554683ce50d8a56: done: 0.027139 sec","job_status":"done","queue_duration_s":0.226023,"scheduling_latency_s":0.226042,"redis_calls":10,"redis_duration_s":0.002564,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001261,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000837,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000466,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027139,"completed_at":"2026-06-29T12:01:48.078Z","load_balancing_strategy":"primary","db_duration_s":0.004308,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2304vm-test-run-nixbot-gitlab> gitlab # [  318.465518] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.080Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"385f5a66185020f9d1c49b61","created_at":"2026-06-29T12:01:47.826Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.828Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-385f5a66185020f9d1c49b61: start","job_status":"start","queue_duration_s":0.251346,"scheduling_latency_s":0.251364}2305vm-test-run-nixbot-gitlab> gitlab # [  318.483083] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:47 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/084809b87ce4cd362622ba41e849f7e129b03d15 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1682306vm-test-run-nixbot-gitlab> gitlab # [  318.485775] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:48 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 1162307vm-test-run-nixbot-gitlab> gitlab # [  318.487885] gitlab-workhorse[1286]: time="2026-06-29T12:01:48Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672308vm-test-run-nixbot-gitlab> gitlab # [  318.489876] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:48 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 842309vm-test-run-nixbot-gitlab> gitlab # [  318.491793] nixbot[3805]: {"ts": "2026-06-29T12:01:47.974Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/084809b87ce4cd362622ba41e849f7e129b03d15 \"HTTP/1.1 201 Created\""}2310vm-test-run-nixbot-gitlab> gitlab # [  318.495166] nix-daemon[4097]: accepted connection from pid 4301, user nixbot2311vm-test-run-nixbot-gitlab> gitlab # [  318.496469] gitaly[1370]: time="2026-06-29T12:01:48.228Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=5 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KW9M3M2DGJS87ZW013WHJ3KH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:48.217" grpc.time_ms=10.566 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc2312vm-test-run-nixbot-gitlab> gitlab # [  318.509669] gitaly[1370]: time="2026-06-29T12:01:48.314Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3M669HH6CBDAZKQ5NDVF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:48.305" pid=1370 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc2313vm-test-run-nixbot-gitlab> gitlab # [  318.521119] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.119Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"385f5a66185020f9d1c49b61","created_at":"2026-06-29T12:01:47.826Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.828Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-385f5a66185020f9d1c49b61: done: 0.039493 sec","job_status":"done","queue_duration_s":0.251346,"scheduling_latency_s":0.251364,"redis_calls":7,"redis_duration_s":0.003478,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002682,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000796,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039493,"completed_at":"2026-06-29T12:01:48.119Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006369,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2314vm-test-run-nixbot-gitlab> gitlab # [  318.564478] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.123Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"55bcf2e5bad0c718de61ba12","created_at":"2026-06-29T12:01:47.829Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.833Z","job_size_bytes":19,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-55bcf2e5bad0c718de61ba12: start","job_status":"start","queue_duration_s":0.290307,"scheduling_latency_s":0.290349}2315vm-test-run-nixbot-gitlab> gitlab # [  318.580921] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.188Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"55bcf2e5bad0c718de61ba12","created_at":"2026-06-29T12:01:47.829Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.833Z","job_size_bytes":19,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-55bcf2e5bad0c718de61ba12: done: 0.064786 sec","job_status":"done","queue_duration_s":0.290307,"scheduling_latency_s":0.290349,"redis_calls":8,"redis_duration_s":0.008805,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004678,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004127,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.064786,"completed_at":"2026-06-29T12:01:48.188Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014384,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2316vm-test-run-nixbot-gitlab> gitlab # [  318.624140] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.194Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"b2ffb0b268cd0cbf9dcd1516","created_at":"2026-06-29T12:01:47.834Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.837Z","job_size_bytes":66,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-b2ffb0b268cd0cbf9dcd1516: start","job_status":"start","queue_duration_s":0.356736,"scheduling_latency_s":0.356757}2317vm-test-run-nixbot-gitlab> gitlab # [  318.641440] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.220Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"b2ffb0b268cd0cbf9dcd1516","created_at":"2026-06-29T12:01:47.834Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.837Z","job_size_bytes":66,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-b2ffb0b268cd0cbf9dcd1516: done: 0.026724 sec","job_status":"done","queue_duration_s":0.356736,"scheduling_latency_s":0.356757,"redis_calls":7,"redis_duration_s":0.0051990000000000005,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001536,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003663,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013859,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026724,"completed_at":"2026-06-29T12:01:48.220Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002715,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2318vm-test-run-nixbot-gitlab> gitlab # [  318.686477] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.229Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"30b22ead5c6ae0520e2cf735","created_at":"2026-06-29T12:01:47.838Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.841Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-30b22ead5c6ae0520e2cf735: start","job_status":"start","queue_duration_s":0.388421,"scheduling_latency_s":0.388441}2319vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/084809b87ce4cd362622ba41e849f7e129b03d15/statuses2320vm-test-run-nixbot-gitlab> gitlab # [  318.703055] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.261Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"30b22ead5c6ae0520e2cf735","created_at":"2026-06-29T12:01:47.838Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.841Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-30b22ead5c6ae0520e2cf735: done: 0.032062 sec","job_status":"done","queue_duration_s":0.388421,"scheduling_latency_s":0.388441,"redis_calls":7,"redis_duration_s":0.0038380000000000003,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000812,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003026,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022469,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032062,"completed_at":"2026-06-29T12:01:48.261Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002925,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2321vm-test-run-nixbot-gitlab> gitlab # [  318.746465] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.263Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"64e1dc1595fdeb1fa771443b","created_at":"2026-06-29T12:01:47.898Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.900Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-64e1dc1595fdeb1fa771443b: start","job_status":"start","queue_duration_s":0.36278,"scheduling_latency_s":0.362803}2322vm-test-run-nixbot-gitlab> gitlab # [  318.761822] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.285Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"64e1dc1595fdeb1fa771443b","created_at":"2026-06-29T12:01:47.898Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.900Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-64e1dc1595fdeb1fa771443b: done: 0.022503 sec","job_status":"done","queue_duration_s":0.36278,"scheduling_latency_s":0.362803,"redis_calls":7,"redis_duration_s":0.000915,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000661,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022503,"completed_at":"2026-06-29T12:01:48.285Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00532,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2323vm-test-run-nixbot-gitlab> gitlab # [  318.804746] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.286Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"d83127c679975a2298c9bb27","created_at":"2026-06-29T12:01:47.900Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.902Z","job_size_bytes":6,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-d83127c679975a2298c9bb27: start","job_status":"start","queue_duration_s":0.383871,"scheduling_latency_s":0.383889}2324vm-test-run-nixbot-gitlab> gitlab # [  318.820687] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.300Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"d83127c679975a2298c9bb27","created_at":"2026-06-29T12:01:47.900Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.902Z","job_size_bytes":6,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-d83127c679975a2298c9bb27: done: 0.01437 sec","job_status":"done","queue_duration_s":0.383871,"scheduling_latency_s":0.383889,"redis_calls":7,"redis_duration_s":0.001727,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001438,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000289,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009161,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01437,"completed_at":"2026-06-29T12:01:48.300Z","load_balancing_strategy":"primary","db_duration_s":0.000331,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2325vm-test-run-nixbot-gitlab> gitlab # [  318.863033] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.302Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"43afd21710c224c9612fd267","created_at":"2026-06-29T12:01:47.903Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.907Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-43afd21710c224c9612fd267: start","job_status":"start","queue_duration_s":0.394619,"scheduling_latency_s":0.394636}2326vm-test-run-nixbot-gitlab> gitlab # [  318.878668] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.334Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"43afd21710c224c9612fd267","created_at":"2026-06-29T12:01:47.903Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.907Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-43afd21710c224c9612fd267: done: 0.032629 sec","job_status":"done","queue_duration_s":0.394619,"scheduling_latency_s":0.394636,"redis_calls":10,"redis_duration_s":0.001557,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000303,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000283,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021374,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026754,"duration_s":0.032629,"completed_at":"2026-06-29T12:01:48.334Z","load_balancing_strategy":"primary","db_duration_s":0.005225,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2327vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/084809b87ce4cd362622ba41e849f7e129b03d15/statuses, in 0.21 seconds)2328vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '084809b87ce4cd362622ba41e849f7e129b03d15', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-29T12:01:42.653Z', 'started_at': None, 'finished_at': '2026-06-29T12:01:47.491Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '084809b87ce4cd362622ba41e849f7e129b03d15', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-06-29T12:01:47.746Z', 'started_at': None, 'finished_at': '2026-06-29T12:01:47.943Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2329vm-test-run-nixbot-gitlab> (finished: subtest: eval and build post commit statuses, in 6.18 seconds)2330vm-test-run-nixbot-gitlab> (finished: run the VM test script, in 384.77 seconds)2331vm-test-run-nixbot-gitlab> gitlab # [  318.922016] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.336Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"745a4952762b3f799f70fd4f","created_at":"2026-06-29T12:01:47.950Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.952Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-745a4952762b3f799f70fd4f: start","job_status":"start","queue_duration_s":0.384207,"scheduling_latency_s":0.384224}2332vm-test-run-nixbot-gitlab> gitlab # [  318.937582] gitaly[1370]: time="2026-06-29T12:01:48.314Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01KW9M3M669HH6CBDAZKQ5NDVF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T12:01:48.305" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1370 remote_ip="2001:db8:1::1" span.kind=server system=grpc2333vm-test-run-nixbot-gitlab> gitlab # [  318.951966] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/29:12:01:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/084809b87ce4cd362622ba41e849f7e129b03d15/statuses HTTP/1.1" 200 1328 "" "curl/8.20.0" 1312334vm-test-run-nixbot-gitlab> gitlab # [  318.954830] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.620Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"745a4952762b3f799f70fd4f","created_at":"2026-06-29T12:01:47.950Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.952Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2415,"job_status":"rescheduled","message":"PipelineProcessWorker JID-745a4952762b3f799f70fd4f: rescheduled"}2335vm-test-run-nixbot-gitlab> test script finished in 384.84s2336vm-test-run-nixbot-gitlab> cleanup2337vm-test-run-nixbot-gitlab> kill QemuMachine (pid 13)2338vm-test-run-nixbot-gitlab> gitlab # [  318.971309] sidekiq-cluster[2415]: {"severity":"INFO","time":"2026-06-29T12:01:48.629Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"745a4952762b3f799f70fd4f","created_at":"2026-06-29T12:01:47.950Z","trace_propagation_headers":{"sentry-trace":"f0d3e726910d4bd4a13cb20e1791a407-a6fb1c811ad545ce","baggage":"sentry-trace_id=f0d3e726910d4bd4a13cb20e1791a407,sentry-release=Unknown"},"correlation_id":"01KW9M3K7DVSWQ3VGN7QANCY86","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T12:01:47.952Z","job_size_bytes":3,"pid":2415,"sidekiq_tid":"m927","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-745a4952762b3f799f70fd4f: done: 0.293052 sec","job_status":"done","queue_duration_s":0.384207,"scheduling_latency_s":0.384224,"gitaly_calls":3,"gitaly_duration_s":0.128358,"redis_calls":68,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.018431,"redis_read_bytes":149,"redis_write_bytes":18168,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000473,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001463,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":22,"redis_queues_duration_s":0.003479,"redis_queues_read_bytes":24,"redis_queues_write_bytes":12310,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.010234,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00021,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000406,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001693,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":26,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":26,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,qemu-system-aarch64: terminating on signal 15 from pid 6 (/nix/store/lqn6mbgzzdrqq2qkwddcmxj9z6amdd86-python3-3.13.13/bin/python3.13)2339vm-test-run-nixbot-gitlab> gitlab # "2340vm-test-run-nixbot-gitlab> (finished: cleanup, in 1.43 seconds)23412342post-build step Upload coverage to codecov: ok2343Skipping codecov: project=Mic92/nixbot attr=aarch64-linux.nixbot-gitlab