1these 13 derivations will be built:2 /nix/store/52hgw0k0fkzgp22paxnx9bkihc25465w-python3.13-nixbot.drv3 /nix/store/6hi6jab2wnb399k8ib40v1lqmdjlnhdd-unit-nixbot.service.drv4 /nix/store/v43xr9v4f55qdxlaw2ai4ippsp344a2a-system-units.drv5 /nix/store/hdd3a63rd7p62sdkgnhvscph22bi3s9j-etc.drv6 /nix/store/8ri133j78q0s3n38hwdfq6c6ck3m2mgj-activate.drv7 /nix/store/lvcl7vsrc0pxhmvj9bjk9sg4kkffy0wj-nixos-system-gitlab-test.drv8 /nix/store/v4c7bbzz88xnz7lvy4mrplbh2pk47kyb-closure-info.drv9 /nix/store/22mp1p38v26c10sdwqqn7bbg0d7a8msm-closure-info.drv10 /nix/store/5hqnikvrwhgwkqsk8hqcx9kxcf19ym9z-run-nixos-vm.drv11 /nix/store/kclynssg26s59klvy1svjfnlf3yf51jr-nixos-vm.drv12 /nix/store/wrq803qzn0yhpifx72j7gmgdfvycgy5a-driverConfiguration.json.drv13 /nix/store/phdxpidbm116zizsi9a3g11x42lxi9y9-nixos-test-driver-nixbot-gitlab.drv14 /nix/store/6gj6v0z8ny43sznfi205babpyaa31s44-vm-test-run-nixbot-gitlab.drv15building '/nix/store/52hgw0k0fkzgp22paxnx9bkihc25465w-python3.13-nixbot.drv' on 'ssh-ng://nix@eliza'16building '/nix/store/52hgw0k0fkzgp22paxnx9bkihc25465w-python3.13-nixbot.drv'17python3.13-nixbot> Sourcing python-remove-tests-dir-hook18python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh19python3.13-nixbot> Sourcing python-remove-bin-bytecode-hook.sh20python3.13-nixbot> Sourcing pypa-build-hook21python3.13-nixbot> Using pypaBuildPhase22python3.13-nixbot> Sourcing python-runtime-deps-check-hook23python3.13-nixbot> Using pythonRuntimeDepsCheckHook24python3.13-nixbot> Sourcing pypa-install-hook25python3.13-nixbot> Using pypaInstallPhase26python3.13-nixbot> Sourcing python-imports-check-hook.sh27python3.13-nixbot> Using pythonImportsCheckPhase28python3.13-nixbot> Sourcing python-namespaces-hook29python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh30python3.13-nixbot> Running phase: unpackPhase31python3.13-nixbot> unpacking source archive /nix/store/7d2k3f14nqrw8r6n8lirk8rsmvp0zh5x-nixbot32python3.13-nixbot> source root is nixbot33python3.13-nixbot> setting SOURCE_DATE_EPOCH to timestamp 315619200 of file "nixbot/pyproject.toml"34python3.13-nixbot> Running phase: patchPhase35python3.13-nixbot> Running phase: updateAutotoolsGnuConfigScriptsPhase36python3.13-nixbot> Running phase: configurePhase37python3.13-nixbot> no configure script, doing nothing38python3.13-nixbot> Running phase: buildPhase39python3.13-nixbot> Executing pypaBuildPhase40python3.13-nixbot> Creating a wheel...41python3.13-nixbot> pypa build flags: --no-isolation --outdir dist/ --wheel42python3.13-nixbot> * Getting build dependencies for wheel...43python3.13-nixbot> * Building wheel...44python3.13-nixbot> Successfully built nixbot-0.0.1-py3-none-any.whl45python3.13-nixbot> Finished creating a wheel...46python3.13-nixbot> Finished executing pypaBuildPhase47python3.13-nixbot> Running phase: pythonRuntimeDepsCheckHook48python3.13-nixbot> Executing pythonRuntimeDepsCheck49python3.13-nixbot> Checking runtime dependencies for nixbot-0.0.1-py3-none-any.whl50python3.13-nixbot> Finished executing pythonRuntimeDepsCheck51python3.13-nixbot> Running phase: installPhase52python3.13-nixbot> Executing pypaInstallPhase53python3.13-nixbot> Successfully installed nixbot-0.0.1-py3-none-any.whl54python3.13-nixbot> Finished executing pypaInstallPhase55python3.13-nixbot> Running phase: pythonOutputDistPhase56python3.13-nixbot> Executing pythonOutputDistPhase57python3.13-nixbot> Finished executing pythonOutputDistPhase58python3.13-nixbot> Running phase: fixupPhase59python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/g4r9fvip9lyv8dvvdphd2plixnhgjqsf-python3.13-nixbot60python3.13-nixbot> checking for references to /build/ in /nix/store/g4r9fvip9lyv8dvvdphd2plixnhgjqsf-python3.13-nixbot...61python3.13-nixbot> patching script interpreter paths in /nix/store/g4r9fvip9lyv8dvvdphd2plixnhgjqsf-python3.13-nixbot62python3.13-nixbot> stripping (with command strip and flags -S -p) in /nix/store/g4r9fvip9lyv8dvvdphd2plixnhgjqsf-python3.13-nixbot/lib /nix/store/g4r9fvip9lyv8dvvdphd2plixnhgjqsf-python3.13-nixbot/bin63python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/pbd7m6fkk7zvsmf4swrw1ax17j92vagg-python3.13-nixbot-dist64python3.13-nixbot> checking for references to /build/ in /nix/store/pbd7m6fkk7zvsmf4swrw1ax17j92vagg-python3.13-nixbot-dist...65python3.13-nixbot> patching script interpreter paths in /nix/store/pbd7m6fkk7zvsmf4swrw1ax17j92vagg-python3.13-nixbot-dist66python3.13-nixbot> Rewriting #!/nix/store/lqn6mbgzzdrqq2qkwddcmxj9z6amdd86-python3-3.13.13/bin/python3.13 to #!/nix/store/lqn6mbgzzdrqq2qkwddcmxj9z6amdd86-python3-3.13.1367python3.13-nixbot> wrapping `/nix/store/g4r9fvip9lyv8dvvdphd2plixnhgjqsf-python3.13-nixbot/bin/nixbot'...68python3.13-nixbot> Executing pythonRemoveTestsDir69python3.13-nixbot> Finished executing pythonRemoveTestsDir70python3.13-nixbot> Running phase: pythonCatchConflictsPhase71python3.13-nixbot> Running phase: pythonRemoveBinBytecodePhase72python3.13-nixbot> Running phase: pythonImportsCheckPhase73python3.13-nixbot> Executing pythonImportsCheckPhase74building '/nix/store/6hi6jab2wnb399k8ib40v1lqmdjlnhdd-unit-nixbot.service.drv'75unit-nixbot.service> tribuchet: building on eliza76building '/nix/store/v43xr9v4f55qdxlaw2ai4ippsp344a2a-system-units.drv'77system-units> tribuchet: building on eliza78building '/nix/store/hdd3a63rd7p62sdkgnhvscph22bi3s9j-etc.drv'79etc> tribuchet: building on eliza80building '/nix/store/8ri133j78q0s3n38hwdfq6c6ck3m2mgj-activate.drv'81activate> tribuchet: building on eliza82building '/nix/store/lvcl7vsrc0pxhmvj9bjk9sg4kkffy0wj-nixos-system-gitlab-test.drv'83nixos-system-gitlab-test> tribuchet: building on eliza84nixos-system-gitlab-test> structuredAttrs is enabled85building '/nix/store/v4c7bbzz88xnz7lvy4mrplbh2pk47kyb-closure-info.drv'86closure-info> tribuchet: building on eliza87closure-info> structuredAttrs is enabled88building '/nix/store/22mp1p38v26c10sdwqqn7bbg0d7a8msm-closure-info.drv'89closure-info> tribuchet: building on eliza90closure-info> structuredAttrs is enabled91building '/nix/store/5hqnikvrwhgwkqsk8hqcx9kxcf19ym9z-run-nixos-vm.drv'92run-nixos-vm> tribuchet: building on eliza93building '/nix/store/kclynssg26s59klvy1svjfnlf3yf51jr-nixos-vm.drv'94nixos-vm> tribuchet: building on eliza95building '/nix/store/wrq803qzn0yhpifx72j7gmgdfvycgy5a-driverConfiguration.json.drv'96driverConfiguration.json> tribuchet: building on eliza97driverConfiguration.json> structuredAttrs is enabled98building '/nix/store/phdxpidbm116zizsi9a3g11x42lxi9y9-nixos-test-driver-nixbot-gitlab.drv'99nixos-test-driver-nixbot-gitlab> tribuchet: building on eliza100nixos-test-driver-nixbot-gitlab> Running type check (enable/disable: config.skipTypeCheck)101nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck102nixos-test-driver-nixbot-gitlab> All checks passed!103nixos-test-driver-nixbot-gitlab> Linting test script (enable/disable: config.skipLint)104nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint105nixos-test-driver-nixbot-gitlab> All checks passed!106building '/nix/store/6gj6v0z8ny43sznfi205babpyaa31s44-vm-test-run-nixbot-gitlab.drv' on 'ssh-ng://nix@eliza'107building '/nix/store/6gj6v0z8ny43sznfi205babpyaa31s44-vm-test-run-nixbot-gitlab.drv'108vm-test-run-nixbot-gitlab> Machine state will be reset. To keep it, pass --keep-machine-state109vm-test-run-nixbot-gitlab> start all VLans110vm-test-run-nixbot-gitlab> (finished: start all VLans, in 0.00 seconds)111vm-test-run-nixbot-gitlab> Test will time out and terminate in 3600 seconds112vm-test-run-nixbot-gitlab> run the VM test script113vm-test-run-nixbot-gitlab> additionally exposed symbols:114vm-test-run-nixbot-gitlab> gitlab,115vm-test-run-nixbot-gitlab> vlan1,116vm-test-run-nixbot-gitlab> start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh117vm-test-run-nixbot-gitlab> start all VMs118vm-test-run-nixbot-gitlab> gitlab: starting vm119vm-test-run-nixbot-gitlab> gitlab: QEMU running (pid 13)120vm-test-run-nixbot-gitlab> gitlab # Disk image does not exist, creating the virtualisation disk image...121vm-test-run-nixbot-gitlab> gitlab # Formatting '/build/vm-state-gitlab/tmp.wpLmKEf90O', fmt=raw size=1073741824122vm-test-run-nixbot-gitlab> gitlab # mke2fs 1.47.3 (8-Jul-2025)123vm-test-run-nixbot-gitlab> gitlab # Discarding device blocks: 0/262144 done124vm-test-run-nixbot-gitlab> gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes125vm-test-run-nixbot-gitlab> gitlab # Filesystem UUID: 05d7ae43-c008-4a40-9046-e98c94ddd7fc126vm-test-run-nixbot-gitlab> gitlab # Superblock backups stored on blocks:127vm-test-run-nixbot-gitlab> gitlab # 32768, 98304, 163840, 229376128vm-test-run-nixbot-gitlab> gitlab # 129vm-test-run-nixbot-gitlab> gitlab # Allocating group tables: 0/8 done130vm-test-run-nixbot-gitlab> gitlab # Writing inode tables: 0/8 done131vm-test-run-nixbot-gitlab> gitlab # Creating journal (8192 blocks): done132vm-test-run-nixbot-gitlab> gitlab # Writing superblocks and filesystem accounting information: 0/8 done133vm-test-run-nixbot-gitlab> gitlab # 134vm-test-run-nixbot-gitlab> gitlab # Virtualisation disk image created.135vm-test-run-nixbot-gitlab> gitlab # Creating Nix store image...136vm-test-run-nixbot-gitlab> gitlab # Created Nix store image.137vm-test-run-nixbot-gitlab> (finished: start all VMs, in 65.25 seconds)138vm-test-run-nixbot-gitlab> gitlab: waiting for unit setup-gitlab.service139vm-test-run-nixbot-gitlab> gitlab: waiting for the VM to finish booting140vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]141vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Jun 1 15:51:08 UTC 2026142vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] KASLR enabled143vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] random: crng init done144vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Machine model: linux,dummy-virt145vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] efi: UEFI not found.146vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT147vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff]148vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf]149vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Zone ranges:150vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]151vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMA32 empty152vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff]153vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Device empty154vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Movable zone start for each node155vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Early memory node ranges156vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff]157vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff]158vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe000000159vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: probing for conduit method from DT.160vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware.161vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs162vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: Trusted OS migration not required163vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: SMC Calling Convention v1.1164vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)165vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s185944 r8192 d117160 u311296166vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Detected PIPT I-cache on CPU0167vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)168vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface169vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: Spectre-v4170vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: Spectre-BHB171vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38172vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23173vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] alternatives: applying boot alternatives174vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/gm1ja6b7ympm67plwx9k8j0agl42j5a4-nixos-system-gitlab-test/init regInfo=/nix/store/qganilc4ln6vli6mb3gsxz3p2fi1v48c-closure-info/registration console=ttyAMA0,115200n8 console=tty0175vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/qganilc4ln6vli6mb3gsxz3p2fi1v48c-closure-info/registration", will be passed to user space.176vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes177vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)178vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)179vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] software IO TLB: area num 4.180vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB)181vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Fallback order for Node 0: 0182vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1572864183vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Policy zone: Normal184vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off185vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1186vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] allocated 12582912 bytes of page_ext187vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ftrace: allocating 74648 entries in 292 pages188vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ftrace: allocated 292 pages with 3 groups189vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: Hierarchical RCU implementation.190vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: RCU event tracing is enabled.191vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.192vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled.193vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Rude variant of Tasks RCU enabled.194vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled.195vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.196vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4197vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.198vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.199vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.200vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0201vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: 256 SPIs implemented202vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented203vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq204vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI205vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0206vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000207vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]208vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f00000 (indirect, esz 8, psz 64K, shr 1)209vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f10000 (flat, esz 8, psz 64K, shr 1)210vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f20000211vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000212vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.213vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).214vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns215vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns216vm-test-run-nixbot-gitlab> gitlab # [ 0.000026] arm-pv: using stolen time PV217vm-test-run-nixbot-gitlab> gitlab # [ 0.000363] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)218vm-test-run-nixbot-gitlab> gitlab # [ 0.000561] Console: colour dummy device 80x25219vm-test-run-nixbot-gitlab> gitlab # [ 0.000568] printk: legacy console [tty0] enabled220vm-test-run-nixbot-gitlab> gitlab # [ 0.000747] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)221vm-test-run-nixbot-gitlab> gitlab # [ 0.000754] pid_max: default: 32768 minimum: 301222vm-test-run-nixbot-gitlab> gitlab # [ 0.000835] LSM: initializing lsm=capability,landlock,yama,bpf,ima223vm-test-run-nixbot-gitlab> gitlab # [ 0.000966] landlock: Up and running.224vm-test-run-nixbot-gitlab> gitlab # [ 0.000969] Yama: becoming mindful.225vm-test-run-nixbot-gitlab> gitlab # [ 0.001376] LSM support for eBPF active226vm-test-run-nixbot-gitlab> gitlab # [ 0.001578] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)227vm-test-run-nixbot-gitlab> gitlab # [ 0.001687] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)228vm-test-run-nixbot-gitlab> gitlab # [ 0.002872] cacheinfo: Unable to detect cache hierarchy for CPU 0229vm-test-run-nixbot-gitlab> gitlab # [ 0.003522] rcu: Hierarchical SRCU implementation.230vm-test-run-nixbot-gitlab> gitlab # [ 0.003526] rcu: Max phase no-delay instances is 1000.231vm-test-run-nixbot-gitlab> gitlab # [ 0.003711] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level232vm-test-run-nixbot-gitlab> gitlab # [ 0.004741] fsl-mc MSI: its@8080000 domain created233vm-test-run-nixbot-gitlab> gitlab # [ 0.004832] EFI services will not be available.234vm-test-run-nixbot-gitlab> gitlab # [ 0.004997] smp: Bringing up secondary CPUs ...235vm-test-run-nixbot-gitlab> gitlab # [ 0.005619] Detected PIPT I-cache on CPU1236vm-test-run-nixbot-gitlab> gitlab # [ 0.005720] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000237vm-test-run-nixbot-gitlab> gitlab # [ 0.005853] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000238vm-test-run-nixbot-gitlab> gitlab # [ 0.005985] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40]239vm-test-run-nixbot-gitlab> gitlab # [ 0.007039] Detected PIPT I-cache on CPU2240vm-test-run-nixbot-gitlab> gitlab # [ 0.007222] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000241vm-test-run-nixbot-gitlab> gitlab # [ 0.007354] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000242vm-test-run-nixbot-gitlab> gitlab # [ 0.007486] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40]243vm-test-run-nixbot-gitlab> gitlab # [ 0.008466] Detected PIPT I-cache on CPU3244vm-test-run-nixbot-gitlab> gitlab # [ 0.008584] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000245vm-test-run-nixbot-gitlab> gitlab # [ 0.008715] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000246vm-test-run-nixbot-gitlab> gitlab # [ 0.008847] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40]247vm-test-run-nixbot-gitlab> gitlab # [ 0.009388] smp: Brought up 1 node, 4 CPUs248vm-test-run-nixbot-gitlab> gitlab # [ 0.009415] SMP: Total of 4 processors activated.249vm-test-run-nixbot-gitlab> gitlab # [ 0.009417] CPU: All CPU(s) started at EL1250vm-test-run-nixbot-gitlab> gitlab # [ 0.009426] CPU features: detected: Branch Target Identification251vm-test-run-nixbot-gitlab> gitlab # [ 0.009429] CPU features: detected: ARMv8.4 Translation Table Level252vm-test-run-nixbot-gitlab> gitlab # [ 0.009432] CPU features: detected: Instruction cache invalidation not required for I/D coherence253vm-test-run-nixbot-gitlab> gitlab # [ 0.009435] CPU features: detected: Data cache clean to the PoU not required for I/D coherence254vm-test-run-nixbot-gitlab> gitlab # [ 0.009438] CPU features: detected: Common not Private translations255vm-test-run-nixbot-gitlab> gitlab # [ 0.009441] CPU features: detected: CRC32 instructions256vm-test-run-nixbot-gitlab> gitlab # [ 0.009444] CPU features: detected: Data cache clean to Point of Deep Persistence257vm-test-run-nixbot-gitlab> gitlab # [ 0.009447] CPU features: detected: Data cache clean to Point of Persistence258vm-test-run-nixbot-gitlab> gitlab # [ 0.009449] CPU features: detected: Data independent timing control (DIT)259vm-test-run-nixbot-gitlab> gitlab # [ 0.009452] CPU features: detected: E0PD260vm-test-run-nixbot-gitlab> gitlab # [ 0.009454] CPU features: detected: Enhanced Counter Virtualization261vm-test-run-nixbot-gitlab> gitlab # [ 0.009457] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)262vm-test-run-nixbot-gitlab> gitlab # [ 0.009460] CPU features: detected: Enhanced Virtualization Traps263vm-test-run-nixbot-gitlab> gitlab # [ 0.009462] CPU features: detected: Fine Grained Traps264vm-test-run-nixbot-gitlab> gitlab # [ 0.009465] CPU features: detected: Generic authentication (architected QARMA5 algorithm)265vm-test-run-nixbot-gitlab> gitlab # [ 0.009469] CPU features: detected: RCpc load-acquire (LDAPR)266vm-test-run-nixbot-gitlab> gitlab # [ 0.009471] CPU features: detected: LSE atomic instructions267vm-test-run-nixbot-gitlab> gitlab # [ 0.009474] CPU features: detected: Privileged Access Never268vm-test-run-nixbot-gitlab> gitlab # [ 0.009476] CPU features: detected: PMUv3269vm-test-run-nixbot-gitlab> gitlab # [ 0.009479] CPU features: detected: RAS Extension Support270vm-test-run-nixbot-gitlab> gitlab # [ 0.009481] CPU features: detected: RASv1p1 Extension Support271vm-test-run-nixbot-gitlab> gitlab # [ 0.009483] CPU features: detected: Random Number Generator272vm-test-run-nixbot-gitlab> gitlab # [ 0.009486] CPU features: detected: Speculation barrier (SB)273vm-test-run-nixbot-gitlab> gitlab # [ 0.009488] CPU features: detected: Stage-2 Force Write-Back274vm-test-run-nixbot-gitlab> gitlab # [ 0.009491] CPU features: detected: TLB range maintenance instructions275vm-test-run-nixbot-gitlab> gitlab # [ 0.009495] CPU features: detected: Speculative Store Bypassing Safe (SSBS)276vm-test-run-nixbot-gitlab> gitlab # [ 0.009626] alternatives: applying system-wide alternatives277vm-test-run-nixbot-gitlab> gitlab # [ 0.012450] CPU features: detected: BBM Level 2 without TLB conflict abort278vm-test-run-nixbot-gitlab> gitlab # [ 0.012615] Memory: 5965768K/6291456K available (24320K kernel code, 7086K rwdata, 26308K rodata, 4736K init, 1102K bss, 270904K reserved, 32768K cma-reserved)279vm-test-run-nixbot-gitlab> gitlab # [ 0.014075] devtmpfs: initialized280vm-test-run-nixbot-gitlab> gitlab # [ 0.017981] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns281vm-test-run-nixbot-gitlab> gitlab # [ 0.018031] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)282vm-test-run-nixbot-gitlab> gitlab # [ 0.018094] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).283vm-test-run-nixbot-gitlab> gitlab # [ 0.018306] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL284vm-test-run-nixbot-gitlab> gitlab # [ 0.018310] 0 pages in range for non-PLT usage285vm-test-run-nixbot-gitlab> gitlab # [ 0.018312] 508336 pages in range for PLT usage286vm-test-run-nixbot-gitlab> gitlab # [ 0.018414] pinctrl core: initialized pinctrl subsystem287vm-test-run-nixbot-gitlab> gitlab # [ 0.019261] DMI not present or invalid.288vm-test-run-nixbot-gitlab> gitlab # [ 0.022418] NET: Registered PF_NETLINK/PF_ROUTE protocol family289vm-test-run-nixbot-gitlab> gitlab # [ 0.025227] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations290vm-test-run-nixbot-gitlab> gitlab # [ 0.025491] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations291vm-test-run-nixbot-gitlab> gitlab # [ 0.027354] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations292vm-test-run-nixbot-gitlab> gitlab # [ 0.027376] audit: initializing netlink subsys (disabled)293vm-test-run-nixbot-gitlab> gitlab # [ 0.027669] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1294vm-test-run-nixbot-gitlab> gitlab # [ 0.028151] thermal_sys: Registered thermal governor 'fair_share'295vm-test-run-nixbot-gitlab> gitlab # [ 0.028154] thermal_sys: Registered thermal governor 'bang_bang'296vm-test-run-nixbot-gitlab> gitlab # [ 0.028159] thermal_sys: Registered thermal governor 'step_wise'297vm-test-run-nixbot-gitlab> gitlab # [ 0.028161] thermal_sys: Registered thermal governor 'user_space'298vm-test-run-nixbot-gitlab> gitlab # [ 0.028164] thermal_sys: Registered thermal governor 'power_allocator'299vm-test-run-nixbot-gitlab> gitlab # [ 0.028243] cpuidle: using governor ladder300vm-test-run-nixbot-gitlab> gitlab # [ 0.028306] cpuidle: using governor menu301vm-test-run-nixbot-gitlab> gitlab # [ 0.028507] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.302vm-test-run-nixbot-gitlab> gitlab # [ 0.028711] ASID allocator initialised with 65536 entries303vm-test-run-nixbot-gitlab> gitlab # [ 0.029820] Serial: AMBA PL011 UART driver304vm-test-run-nixbot-gitlab> gitlab # [ 0.034834] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1305vm-test-run-nixbot-gitlab> gitlab # [ 0.035165] printk: console [ttyAMA0] enabled306vm-test-run-nixbot-gitlab> gitlab # [ 0.051152] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages307vm-test-run-nixbot-gitlab> gitlab # [ 0.051174] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page308vm-test-run-nixbot-gitlab> gitlab # [ 0.051184] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages309vm-test-run-nixbot-gitlab> gitlab # [ 0.051192] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page310vm-test-run-nixbot-gitlab> gitlab # [ 0.051200] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages311vm-test-run-nixbot-gitlab> gitlab # [ 0.051208] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page312vm-test-run-nixbot-gitlab> gitlab # [ 0.051216] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages313vm-test-run-nixbot-gitlab> gitlab # [ 0.051224] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page314vm-test-run-nixbot-gitlab> gitlab # [ 0.062887] fbcon: Taking over console315vm-test-run-nixbot-gitlab> gitlab # [ 0.062898] ACPI: Interpreter disabled.316vm-test-run-nixbot-gitlab> gitlab # [ 0.063856] iommu: Default domain type: Translated317vm-test-run-nixbot-gitlab> gitlab # [ 0.063860] iommu: DMA domain TLB invalidation policy: strict mode318vm-test-run-nixbot-gitlab> gitlab # [ 0.073871] SCSI subsystem initialized319vm-test-run-nixbot-gitlab> gitlab # [ 0.075059] usbcore: registered new interface driver usbfs320vm-test-run-nixbot-gitlab> gitlab # [ 0.075188] usbcore: registered new interface driver hub321vm-test-run-nixbot-gitlab> gitlab # [ 0.075275] usbcore: registered new device driver usb322vm-test-run-nixbot-gitlab> gitlab # [ 0.076142] pps_core: LinuxPPS API ver. 1 registered323vm-test-run-nixbot-gitlab> gitlab # [ 0.076153] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>324vm-test-run-nixbot-gitlab> gitlab # [ 0.076176] PTP clock support registered325vm-test-run-nixbot-gitlab> gitlab # [ 0.076276] EDAC MC: Ver: 3.0.0326vm-test-run-nixbot-gitlab> gitlab # [ 0.076691] scmi_core: SCMI protocol bus registered327vm-test-run-nixbot-gitlab> gitlab # [ 0.077593] FPGA manager framework328vm-test-run-nixbot-gitlab> gitlab # [ 0.079729] vgaarb: loaded329vm-test-run-nixbot-gitlab> gitlab # [ 0.080739] clocksource: Switched to clocksource arch_sys_counter330vm-test-run-nixbot-gitlab> gitlab # [ 0.081264] VFS: Disk quotas dquot_6.6.0331vm-test-run-nixbot-gitlab> gitlab # [ 0.081290] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)332vm-test-run-nixbot-gitlab> gitlab # [ 0.081644] netfs: FS-Cache loaded333vm-test-run-nixbot-gitlab> gitlab # [ 0.081754] pnp: PnP ACPI: disabled334vm-test-run-nixbot-gitlab> gitlab # [ 0.088225] NET: Registered PF_INET protocol family335vm-test-run-nixbot-gitlab> gitlab # [ 0.089317] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)336vm-test-run-nixbot-gitlab> gitlab # [ 0.118802] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)337vm-test-run-nixbot-gitlab> gitlab # [ 0.118853] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)338vm-test-run-nixbot-gitlab> gitlab # [ 0.118894] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)339vm-test-run-nixbot-gitlab> gitlab # [ 0.119177] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)340vm-test-run-nixbot-gitlab> gitlab # [ 0.119726] TCP: Hash tables configured (established 65536 bind 65536)341vm-test-run-nixbot-gitlab> gitlab # [ 0.119807] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)342vm-test-run-nixbot-gitlab> gitlab # [ 0.119856] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)343vm-test-run-nixbot-gitlab> gitlab # [ 0.120011] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)344vm-test-run-nixbot-gitlab> gitlab # [ 0.120187] NET: Registered PF_UNIX/PF_LOCAL protocol family345vm-test-run-nixbot-gitlab> gitlab # [ 0.120236] NET: Registered PF_XDP protocol family346vm-test-run-nixbot-gitlab> gitlab # [ 0.120249] PCI: CLS 0 bytes, default 64347vm-test-run-nixbot-gitlab> gitlab # [ 0.120651] Trying to unpack rootfs image as initramfs...348vm-test-run-nixbot-gitlab> gitlab # [ 0.133804] kvm [1]: HYP mode not available349vm-test-run-nixbot-gitlab> gitlab # [ 0.219384] Initialise system trusted keyrings350vm-test-run-nixbot-gitlab> gitlab # [ 0.219702] workingset: timestamp_bits=42 max_order=21 bucket_order=0351vm-test-run-nixbot-gitlab> gitlab # [ 0.220615] squashfs: version 4.0 (2009/01/31) Phillip Lougher352vm-test-run-nixbot-gitlab> gitlab # [ 0.220682] 9p: Installing v9fs 9p2000 file system support353vm-test-run-nixbot-gitlab> gitlab # [ 0.237462] Key type asymmetric registered354vm-test-run-nixbot-gitlab> gitlab # [ 0.237468] Asymmetric key parser 'x509' registered355vm-test-run-nixbot-gitlab> gitlab # [ 0.237588] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)356vm-test-run-nixbot-gitlab> gitlab # [ 0.237942] io scheduler mq-deadline registered357vm-test-run-nixbot-gitlab> gitlab # [ 0.237949] io scheduler kyber registered358vm-test-run-nixbot-gitlab> gitlab # [ 0.252401] pl061_gpio 9030000.pl061: PL061 GPIO chip registered359vm-test-run-nixbot-gitlab> gitlab # [ 0.260865] ledtrig-cpu: registered to indicate activity on CPUs360vm-test-run-nixbot-gitlab> gitlab # [ 0.261683] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:361vm-test-run-nixbot-gitlab> gitlab # [ 0.261739] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000362vm-test-run-nixbot-gitlab> gitlab # [ 0.261761] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000363vm-test-run-nixbot-gitlab> gitlab # [ 0.261775] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000364vm-test-run-nixbot-gitlab> gitlab # [ 0.261812] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits365vm-test-run-nixbot-gitlab> gitlab # [ 0.261863] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]366vm-test-run-nixbot-gitlab> gitlab # [ 0.262071] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00367vm-test-run-nixbot-gitlab> gitlab # [ 0.262085] pci_bus 0000:00: root bus resource [bus 00-ff]368vm-test-run-nixbot-gitlab> gitlab # [ 0.262093] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]369vm-test-run-nixbot-gitlab> gitlab # [ 0.262100] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]370vm-test-run-nixbot-gitlab> gitlab # [ 0.262106] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]371vm-test-run-nixbot-gitlab> gitlab # [ 0.262304] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint372vm-test-run-nixbot-gitlab> gitlab # [ 0.263304] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint373vm-test-run-nixbot-gitlab> gitlab # [ 0.263725] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]374vm-test-run-nixbot-gitlab> gitlab # [ 0.263751] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]375vm-test-run-nixbot-gitlab> gitlab # [ 0.263812] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]376vm-test-run-nixbot-gitlab> gitlab # [ 0.263839] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]377vm-test-run-nixbot-gitlab> gitlab # [ 0.265010] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint378vm-test-run-nixbot-gitlab> gitlab # [ 0.265444] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]379vm-test-run-nixbot-gitlab> gitlab # [ 0.265470] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]380vm-test-run-nixbot-gitlab> gitlab # [ 0.265535] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]381vm-test-run-nixbot-gitlab> gitlab # [ 0.266633] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint382vm-test-run-nixbot-gitlab> gitlab # [ 0.267063] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f]383vm-test-run-nixbot-gitlab> gitlab # [ 0.267089] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]384vm-test-run-nixbot-gitlab> gitlab # [ 0.267155] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]385vm-test-run-nixbot-gitlab> gitlab # [ 0.268225] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint386vm-test-run-nixbot-gitlab> gitlab # [ 0.268652] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]387vm-test-run-nixbot-gitlab> gitlab # [ 0.268677] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]388vm-test-run-nixbot-gitlab> gitlab # [ 0.268762] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]389vm-test-run-nixbot-gitlab> gitlab # [ 0.269882] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint390vm-test-run-nixbot-gitlab> gitlab # [ 0.270316] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f]391vm-test-run-nixbot-gitlab> gitlab # [ 0.270342] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]392vm-test-run-nixbot-gitlab> gitlab # [ 0.270408] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]393vm-test-run-nixbot-gitlab> gitlab # [ 0.271482] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint394vm-test-run-nixbot-gitlab> gitlab # [ 0.271904] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]395vm-test-run-nixbot-gitlab> gitlab # [ 0.271930] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]396vm-test-run-nixbot-gitlab> gitlab # [ 0.271989] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]397vm-test-run-nixbot-gitlab> gitlab # [ 0.273108] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint398vm-test-run-nixbot-gitlab> gitlab # [ 0.273551] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]399vm-test-run-nixbot-gitlab> gitlab # [ 0.273577] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]400vm-test-run-nixbot-gitlab> gitlab # [ 0.273636] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]401vm-test-run-nixbot-gitlab> gitlab # [ 0.273662] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]402vm-test-run-nixbot-gitlab> gitlab # [ 0.274838] pci 0000:00:08.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint403vm-test-run-nixbot-gitlab> gitlab # [ 0.275274] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]404vm-test-run-nixbot-gitlab> gitlab # [ 0.275333] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]405vm-test-run-nixbot-gitlab> gitlab # [ 0.276417] pci 0000:00:09.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint406vm-test-run-nixbot-gitlab> gitlab # [ 0.276886] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]407vm-test-run-nixbot-gitlab> gitlab # [ 0.276946] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]408vm-test-run-nixbot-gitlab> gitlab # [ 0.277898] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint409vm-test-run-nixbot-gitlab> gitlab # [ 0.278331] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]410vm-test-run-nixbot-gitlab> gitlab # [ 0.278899] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint411vm-test-run-nixbot-gitlab> gitlab # [ 0.279386] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]412vm-test-run-nixbot-gitlab> gitlab # [ 0.279413] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]413vm-test-run-nixbot-gitlab> gitlab # [ 0.279471] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]414vm-test-run-nixbot-gitlab> gitlab # [ 0.280557] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint415vm-test-run-nixbot-gitlab> gitlab # [ 0.281001] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]416vm-test-run-nixbot-gitlab> gitlab # [ 0.281027] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]417vm-test-run-nixbot-gitlab> gitlab # [ 0.281086] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]418vm-test-run-nixbot-gitlab> gitlab # [ 0.282468] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned419vm-test-run-nixbot-gitlab> gitlab # [ 0.282482] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned420vm-test-run-nixbot-gitlab> gitlab # [ 0.282490] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned421vm-test-run-nixbot-gitlab> gitlab # [ 0.282588] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned422vm-test-run-nixbot-gitlab> gitlab # [ 0.282685] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned423vm-test-run-nixbot-gitlab> gitlab # [ 0.282792] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned424vm-test-run-nixbot-gitlab> gitlab # [ 0.282892] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned425vm-test-run-nixbot-gitlab> gitlab # [ 0.283003] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned426vm-test-run-nixbot-gitlab> gitlab # [ 0.283103] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned427vm-test-run-nixbot-gitlab> gitlab # [ 0.283205] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned428vm-test-run-nixbot-gitlab> gitlab # [ 0.283314] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned429vm-test-run-nixbot-gitlab> gitlab # [ 0.283422] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned430vm-test-run-nixbot-gitlab> gitlab # [ 0.283560] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned431vm-test-run-nixbot-gitlab> gitlab # [ 0.283657] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned432vm-test-run-nixbot-gitlab> gitlab # [ 0.283698] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned433vm-test-run-nixbot-gitlab> gitlab # [ 0.283737] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned434vm-test-run-nixbot-gitlab> gitlab # [ 0.283778] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned435vm-test-run-nixbot-gitlab> gitlab # [ 0.283828] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned436vm-test-run-nixbot-gitlab> gitlab # [ 0.283868] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned437vm-test-run-nixbot-gitlab> gitlab # [ 0.283910] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned438vm-test-run-nixbot-gitlab> gitlab # [ 0.283959] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned439vm-test-run-nixbot-gitlab> gitlab # [ 0.284000] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned440vm-test-run-nixbot-gitlab> gitlab # [ 0.284041] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned441vm-test-run-nixbot-gitlab> gitlab # [ 0.284090] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned442vm-test-run-nixbot-gitlab> gitlab # [ 0.284134] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned443vm-test-run-nixbot-gitlab> gitlab # [ 0.284172] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned444vm-test-run-nixbot-gitlab> gitlab # [ 0.284211] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned445vm-test-run-nixbot-gitlab> gitlab # [ 0.284250] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned446vm-test-run-nixbot-gitlab> gitlab # [ 0.284289] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned447vm-test-run-nixbot-gitlab> gitlab # [ 0.284329] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned448vm-test-run-nixbot-gitlab> gitlab # [ 0.284370] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned449vm-test-run-nixbot-gitlab> gitlab # [ 0.284410] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned450vm-test-run-nixbot-gitlab> gitlab # [ 0.284450] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned451vm-test-run-nixbot-gitlab> gitlab # [ 0.284491] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned452vm-test-run-nixbot-gitlab> gitlab # [ 0.284541] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]453vm-test-run-nixbot-gitlab> gitlab # [ 0.284552] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]454vm-test-run-nixbot-gitlab> gitlab # [ 0.284558] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]455vm-test-run-nixbot-gitlab> gitlab # [ 0.287336] pci 0000:00:0a.0: enabling device (0000 -> 0002)456vm-test-run-nixbot-gitlab> gitlab # [ 0.309373] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)457vm-test-run-nixbot-gitlab> gitlab # [ 0.312905] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)458vm-test-run-nixbot-gitlab> gitlab # [ 0.316155] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)459vm-test-run-nixbot-gitlab> gitlab # [ 0.319551] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)460vm-test-run-nixbot-gitlab> gitlab # [ 0.325955] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)461vm-test-run-nixbot-gitlab> gitlab # [ 0.329387] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)462vm-test-run-nixbot-gitlab> gitlab # [ 0.332819] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)463vm-test-run-nixbot-gitlab> gitlab # [ 0.337793] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)464vm-test-run-nixbot-gitlab> gitlab # [ 0.341877] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)465vm-test-run-nixbot-gitlab> gitlab # [ 0.345245] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)466vm-test-run-nixbot-gitlab> gitlab # [ 0.349139] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)467vm-test-run-nixbot-gitlab> gitlab # [ 0.360277] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled468vm-test-run-nixbot-gitlab> gitlab # [ 0.363119] msm_serial: driver initialized469vm-test-run-nixbot-gitlab> gitlab # [ 0.363412] SuperH (H)SCI(F) driver initialized470vm-test-run-nixbot-gitlab> gitlab # [ 0.363494] STM32 USART driver initialized471vm-test-run-nixbot-gitlab> gitlab # [ 0.393857] loop: module loaded472vm-test-run-nixbot-gitlab> gitlab # [ 0.394176] virtio_blk virtio4: 4/0/0 default/read/poll queues473vm-test-run-nixbot-gitlab> gitlab # [ 0.397125] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)474vm-test-run-nixbot-gitlab> gitlab # [ 0.399781] virtio_blk virtio5: 4/0/0 default/read/poll queues475vm-test-run-nixbot-gitlab> gitlab # [ 0.402350] virtio_blk virtio5: [vdb] 11827216 512-byte logical blocks (6.06 GB/5.64 GiB)476vm-test-run-nixbot-gitlab> gitlab # [ 0.405120] megasas: 07.734.00.00-rc1477vm-test-run-nixbot-gitlab> gitlab # [ 0.406136] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]478vm-test-run-nixbot-gitlab> gitlab # [ 0.408950] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000479vm-test-run-nixbot-gitlab> gitlab # [ 0.408969] Intel/Sharp Extended Query Table at 0x0031480vm-test-run-nixbot-gitlab> gitlab # [ 0.411291] Using buffer write method481vm-test-run-nixbot-gitlab> gitlab # [ 0.411328] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]482vm-test-run-nixbot-gitlab> gitlab # [ 0.414029] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000483vm-test-run-nixbot-gitlab> gitlab # [ 0.414050] Intel/Sharp Extended Query Table at 0x0031484vm-test-run-nixbot-gitlab> gitlab # [ 0.416564] Using buffer write method485vm-test-run-nixbot-gitlab> gitlab # [ 0.416585] Concatenating MTD devices:486vm-test-run-nixbot-gitlab> gitlab # [ 0.416588] (0): "0.flash"487vm-test-run-nixbot-gitlab> gitlab # [ 0.416591] (1): "0.flash"488vm-test-run-nixbot-gitlab> gitlab # [ 0.416594] into device "0.flash"489vm-test-run-nixbot-gitlab> gitlab # [ 0.593781] Freeing initrd memory: 25372K490vm-test-run-nixbot-gitlab> gitlab # [ 0.612242] tun: Universal TUN/TAP device driver, 1.6491vm-test-run-nixbot-gitlab> gitlab # [ 0.621335] thunder_xcv, ver 1.0492vm-test-run-nixbot-gitlab> gitlab # [ 0.621446] thunder_bgx, ver 1.0493vm-test-run-nixbot-gitlab> gitlab # [ 0.621508] nicpf, ver 1.0494vm-test-run-nixbot-gitlab> gitlab # [ 0.623282] e1000: Intel(R) PRO/1000 Network Driver495vm-test-run-nixbot-gitlab> gitlab # [ 0.623295] e1000: Copyright (c) 1999-2006 Intel Corporation.496vm-test-run-nixbot-gitlab> gitlab # [ 0.623363] e1000e: Intel(R) PRO/1000 Network Driver497vm-test-run-nixbot-gitlab> gitlab # [ 0.623380] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.498vm-test-run-nixbot-gitlab> gitlab # [ 0.623462] igb: Intel(R) Gigabit Ethernet Network Driver499vm-test-run-nixbot-gitlab> gitlab # [ 0.623471] igb: Copyright (c) 2007-2014 Intel Corporation.500vm-test-run-nixbot-gitlab> gitlab # [ 0.623536] igbvf: Intel(R) Gigabit Virtual Function Network Driver501vm-test-run-nixbot-gitlab> gitlab # [ 0.623546] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.502vm-test-run-nixbot-gitlab> gitlab # [ 0.624014] sky2: driver version 1.30503vm-test-run-nixbot-gitlab> gitlab # [ 0.628996] usbcore: registered new interface driver usb-storage504vm-test-run-nixbot-gitlab> gitlab # [ 0.629702] ehci-pci 0000:00:0a.0: EHCI Host Controller505vm-test-run-nixbot-gitlab> gitlab # [ 0.629778] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1506vm-test-run-nixbot-gitlab> gitlab # [ 0.629803] usbcore: registered new interface driver usbserial_generic507vm-test-run-nixbot-gitlab> gitlab # [ 0.629846] usbserial: USB Serial support registered for generic508vm-test-run-nixbot-gitlab> gitlab # [ 0.630486] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000509vm-test-run-nixbot-gitlab> gitlab # [ 0.631765] hv_vmbus: registering driver hyperv_keyboard510vm-test-run-nixbot-gitlab> gitlab # [ 0.634345] rtc-pl031 9010000.pl031: registered as rtc0511vm-test-run-nixbot-gitlab> gitlab # [ 0.634412] rtc-pl031 9010000.pl031: setting system clock to 2026-07-01T08:05:26 UTC (1782893126)512vm-test-run-nixbot-gitlab> gitlab # [ 0.635363] i2c_dev: i2c /dev entries driver513vm-test-run-nixbot-gitlab> gitlab # [ 0.640847] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00514vm-test-run-nixbot-gitlab> gitlab # [ 0.642099] hub 1-0:1.0: USB hub found515vm-test-run-nixbot-gitlab> gitlab # [ 0.642213] hub 1-0:1.0: 6 ports detected516vm-test-run-nixbot-gitlab> gitlab # [ 0.644006] sdhci: Secure Digital Host Controller Interface driver517vm-test-run-nixbot-gitlab> gitlab # [ 0.644021] sdhci: Copyright(c) Pierre Ossman518vm-test-run-nixbot-gitlab> gitlab # [ 0.644920] Synopsys Designware Multimedia Card Interface Driver519vm-test-run-nixbot-gitlab> gitlab # [ 0.646108] sdhci-pltfm: SDHCI platform and OF driver helper520vm-test-run-nixbot-gitlab> gitlab # [ 0.650965] hid: raw HID events driver (C) Jiri Kosina521vm-test-run-nixbot-gitlab> gitlab # [ 0.651861] usbcore: registered new interface driver usbhid522vm-test-run-nixbot-gitlab> gitlab # [ 0.651872] usbhid: USB HID core driver523vm-test-run-nixbot-gitlab> gitlab # [ 0.682528] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available524vm-test-run-nixbot-gitlab> gitlab # [ 0.687356] drop_monitor: Initializing network drop monitor service525vm-test-run-nixbot-gitlab> gitlab # [ 0.687773] NET: Registered PF_INET6 protocol family526vm-test-run-nixbot-gitlab> gitlab # [ 0.690277] Segment Routing with IPv6527vm-test-run-nixbot-gitlab> gitlab # [ 0.690349] In-situ OAM (IOAM) with IPv6528vm-test-run-nixbot-gitlab> gitlab # [ 0.690610] NET: Registered PF_PACKET protocol family529vm-test-run-nixbot-gitlab> gitlab # [ 0.691046] 9pnet: Installing 9P2000 support530vm-test-run-nixbot-gitlab> gitlab # [ 0.695472] Key type dns_resolver registered531vm-test-run-nixbot-gitlab> gitlab # [ 0.714487] registered taskstats version 1532vm-test-run-nixbot-gitlab> gitlab # [ 0.715031] Loading compiled-in X.509 certificates533vm-test-run-nixbot-gitlab> gitlab # [ 0.738340] Demotion targets for Node 0: null534vm-test-run-nixbot-gitlab> gitlab # [ 0.738572] Key type .fscrypt registered535vm-test-run-nixbot-gitlab> gitlab # [ 0.738580] Key type fscrypt-provisioning registered536vm-test-run-nixbot-gitlab> gitlab # [ 0.738758] ima: No TPM chip found, activating TPM-bypass!537vm-test-run-nixbot-gitlab> gitlab # [ 0.738781] ima: Allocated hash algorithm: sha1538vm-test-run-nixbot-gitlab> gitlab # [ 0.738827] ima: No architecture policies found539vm-test-run-nixbot-gitlab> gitlab # [ 0.739915] input: gpio-keys as /devices/platform/gpio-keys/input/input0540vm-test-run-nixbot-gitlab> gitlab # [ 0.768590] clk: Disabling unused clocks541vm-test-run-nixbot-gitlab> gitlab # [ 0.768620] PM: genpd: Disabling unused power domains542vm-test-run-nixbot-gitlab> gitlab # [ 0.774316] Freeing unused kernel memory: 4736K543vm-test-run-nixbot-gitlab> gitlab # [ 0.774552] Run /init as init process544vm-test-run-nixbot-gitlab> gitlab # [ 0.807484] systemd[1]: Successfully made /usr/ read-only.545vm-test-run-nixbot-gitlab> gitlab # [ 0.888867] usb 1-1: new high-speed USB device number 2 using ehci-pci546vm-test-run-nixbot-gitlab> gitlab # [ 1.041373] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1547vm-test-run-nixbot-gitlab> gitlab # [ 1.129425] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0548vm-test-run-nixbot-gitlab> gitlab # [ 1.142060] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)549vm-test-run-nixbot-gitlab> gitlab # [ 1.142125] systemd[1]: Detected virtualization qemu.550vm-test-run-nixbot-gitlab> gitlab # [ 1.142198] systemd[1]: Detected architecture arm64.551vm-test-run-nixbot-gitlab> gitlab # [ 1.142214] systemd[1]: Running in initrd.552vm-test-run-nixbot-gitlab> gitlab # [ 1.143188] systemd[1]: Initializing machine ID from random generator.553vm-test-run-nixbot-gitlab> gitlab # [ 1.143437] systemd[1]: Hostname set to <gitlab>.554vm-test-run-nixbot-gitlab> gitlab # [ 1.256830] usb 1-2: new high-speed USB device number 3 using ehci-pci555vm-test-run-nixbot-gitlab> gitlab # [ 1.288026] systemd[1]: Queued start job for default target Initrd Default Target.556vm-test-run-nixbot-gitlab> gitlab # [ 1.419434] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2557vm-test-run-nixbot-gitlab> gitlab # [ 1.419967] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0558vm-test-run-nixbot-gitlab> gitlab # [ 1.529665] systemd[1]: Created slice Slice /system/modprobe.559vm-test-run-nixbot-gitlab> gitlab # [ 1.530166] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.560vm-test-run-nixbot-gitlab> gitlab # [ 1.530276] systemd[1]: Expecting device /dev/disk/by-label/nix-store...561vm-test-run-nixbot-gitlab> gitlab # [ 1.530309] systemd[1]: Expecting device /dev/disk/by-label/nixos...562vm-test-run-nixbot-gitlab> gitlab # [ 1.530362] systemd[1]: Reached target Path Units.563vm-test-run-nixbot-gitlab> gitlab # [ 1.530401] systemd[1]: Reached target Slice Units.564vm-test-run-nixbot-gitlab> gitlab # [ 1.530442] systemd[1]: Reached target Swaps.565vm-test-run-nixbot-gitlab> gitlab # [ 1.530481] systemd[1]: Reached target Timer Units.566vm-test-run-nixbot-gitlab> gitlab # [ 1.530811] systemd[1]: Listening on D-Bus System Message Bus Socket.567vm-test-run-nixbot-gitlab> gitlab # [ 1.531134] systemd[1]: Listening on Journal Socket (/dev/log).568vm-test-run-nixbot-gitlab> gitlab # [ 1.531531] systemd[1]: Listening on Journal Sockets.569vm-test-run-nixbot-gitlab> gitlab # [ 1.531833] systemd[1]: Listening on udev Control Socket.570vm-test-run-nixbot-gitlab> gitlab # [ 1.532007] systemd[1]: Listening on udev Kernel Socket.571vm-test-run-nixbot-gitlab> gitlab # [ 1.532049] systemd[1]: Reached target Socket Units.572vm-test-run-nixbot-gitlab> gitlab # [ 1.539372] systemd[1]: Starting Create List of Static Device Nodes...573vm-test-run-nixbot-gitlab> gitlab # [ 1.547254] systemd[1]: Starting Load Kernel Module 9pnet_virtio...574vm-test-run-nixbot-gitlab> gitlab # [ 1.547402] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs575vm-test-run-nixbot-gitlab> gitlab # [ 1.573724] systemd[1]: Mounting Kernel Configuration File System...576vm-test-run-nixbot-gitlab> gitlab # [ 1.586299] systemd[1]: Starting Journal Service...577vm-test-run-nixbot-gitlab> gitlab # [ 1.594301] systemd[1]: Starting Load Kernel Modules...578vm-test-run-nixbot-gitlab> gitlab # [ 1.594485] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki579vm-test-run-nixbot-gitlab> gitlab # [ 1.601938] systemd[1]: Starting Coldplug All udev Devices...580vm-test-run-nixbot-gitlab> gitlab # [ 1.637465] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.581vm-test-run-nixbot-gitlab> gitlab # [ 1.638072] systemd[1]: Finished Create List of Static Device Nodes.582vm-test-run-nixbot-gitlab> gitlab # [ 1.638084] [drm] pci: virtio-gpu-pci detected at 0000:00:09.0583vm-test-run-nixbot-gitlab> gitlab # [ 1.638755] [drm] features: -virgl +edid -resource_blob -host_visible584vm-test-run-nixbot-gitlab> gitlab # [ 1.638783] [drm] features: -context_init585vm-test-run-nixbot-gitlab> gitlab # [ 1.639728] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.586vm-test-run-nixbot-gitlab> gitlab # [ 1.641090] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev587vm-test-run-nixbot-gitlab> gitlab # [ 1.643536] systemd[1]: Finished Load Kernel Module 9pnet_virtio.588vm-test-run-nixbot-gitlab> gitlab # [ 1.644677] systemd-journald[154]: Collecting audit messages is disabled.589vm-test-run-nixbot-gitlab> gitlab # [ 1.646563] [drm] number of scanouts: 1590vm-test-run-nixbot-gitlab> gitlab # [ 1.646594] [drm] number of cap sets: 0591vm-test-run-nixbot-gitlab> gitlab # [ 1.649109] virtio-pci 0000:00:09.0: [drm] Registered 1 planes with drm panic592vm-test-run-nixbot-gitlab> gitlab # [ 1.649123] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:09.0 on minor 0593vm-test-run-nixbot-gitlab> gitlab # [ 1.653170] systemd[1]: Mounted Kernel Configuration File System.594vm-test-run-nixbot-gitlab> gitlab # [ 1.664650] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...595vm-test-run-nixbot-gitlab> gitlab # [ 1.669276] Console: switching to colour frame buffer device 160x50596vm-test-run-nixbot-gitlab> gitlab # [ 1.675869] virtio-pci 0000:00:09.0: [drm] fb0: virtio_gpudrmfb frame buffer device597vm-test-run-nixbot-gitlab> gitlab # [ 1.696202] systemd[1]: Finished Load Kernel Modules.598vm-test-run-nixbot-gitlab> gitlab # [ 1.705384] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.599vm-test-run-nixbot-gitlab> gitlab # [ 1.710550] systemd[1]: Starting Apply Kernel Variables...600vm-test-run-nixbot-gitlab> gitlab # [ 1.721202] systemd[1]: Starting Create Static Device Nodes in /dev...601vm-test-run-nixbot-gitlab> gitlab # [ 1.724886] systemd[1]: Finished Apply Kernel Variables.602vm-test-run-nixbot-gitlab> gitlab # [ 1.746147] systemd[1]: Finished Create Static Device Nodes in /dev.603vm-test-run-nixbot-gitlab> gitlab # [ 1.746537] systemd[1]: Reached target Preparation for Local File Systems.604vm-test-run-nixbot-gitlab> gitlab # [ 1.747069] systemd[1]: Reached target Local File Systems.605vm-test-run-nixbot-gitlab> gitlab # [ 1.743697] systemd-modules-load[157]: Using 4 probe threads606vm-test-run-nixbot-gitlab> gitlab # [ 1.749290] systemd-modules-load[157]: Module 'virtio_balloon' is built in607vm-test-run-nixbot-gitlab> gitlab # [ 1.750542] systemd-modules-load[157]: Module 'virtio_console' is built in[ 1.754945] systemd[1]: Starting Rule-based Manager for Device Events and Files...608vm-test-run-nixbot-gitlab> gitlab # 609vm-test-run-nixbot-gitlab> gitlab # [ 1.755401] systemd[1]: Started Journal Service.610vm-test-run-nixbot-gitlab> gitlab # [ 1.756382] systemd-modules-load[157]: Module 'virtio_rng' is built in611vm-test-run-nixbot-gitlab> gitlab # [ 1.757600] systemd-modules-load[157]: Inserted module 'dm_mod'612vm-test-run-nixbot-gitlab> gitlab # [ 1.758700] systemd-modules-load[157]: Inserted module 'virtio_gpu'613vm-test-run-nixbot-gitlab> gitlab # [ 1.761666] systemd[1]: Starting Create System Files and Directories...614vm-test-run-nixbot-gitlab> gitlab # [ 1.800042] systemd[1]: Finished Create System Files and Directories.615vm-test-run-nixbot-gitlab> gitlab # [ 1.826908] systemd-udevd[181]: Using default interface naming scheme 'v260'.616vm-test-run-nixbot-gitlab> gitlab # [ 1.852613] systemd[1]: Started Rule-based Manager for Device Events and Files.617vm-test-run-nixbot-gitlab> gitlab # [ 1.883619] systemd[1]: Starting Virtual Console Setup...618vm-test-run-nixbot-gitlab> gitlab # [ 1.941120] systemd-vconsole-setup[211]: Configuration of first virtual console was skipped, ignoring remaining ones.619vm-test-run-nixbot-gitlab> gitlab # [ 1.949227] systemd[1]: Finished Virtual Console Setup.620vm-test-run-nixbot-gitlab> gitlab # [ 2.379924] systemd[1]: Finished Coldplug All udev Devices.621vm-test-run-nixbot-gitlab> gitlab # [ 2.382513] systemd[1]: Reached target System Initialization.622vm-test-run-nixbot-gitlab> gitlab # [ 2.385135] systemd[1]: Reached target Basic System.623vm-test-run-nixbot-gitlab> gitlab # [ 2.611590] systemd[1]: Found device /dev/disk/by-label/nix-store.624vm-test-run-nixbot-gitlab> gitlab # [ 2.616172] (udev-worker)[220]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.625vm-test-run-nixbot-gitlab> gitlab # [ 2.620241] (udev-worker)[220]: Network interface NamePolicy= disabled on kernel command line.626vm-test-run-nixbot-gitlab> gitlab # [ 2.630314] (udev-worker)[227]: Network interface NamePolicy= disabled on kernel command line.627vm-test-run-nixbot-gitlab> gitlab # [ 2.632077] systemd[1]: Found device /dev/disk/by-label/nixos.628vm-test-run-nixbot-gitlab> gitlab # [ 2.633279] systemd[1]: Reached target Initrd Root Device.629vm-test-run-nixbot-gitlab> gitlab # [ 2.639408] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...630vm-test-run-nixbot-gitlab> gitlab # [ 2.679132] systemd-fsck[236]: nixos: clean, 12/65536 files, 13019/262144 blocks631vm-test-run-nixbot-gitlab> gitlab # [ 2.685389] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.632vm-test-run-nixbot-gitlab> gitlab # [ 2.689681] systemd[1]: Mounting /sysroot...633vm-test-run-nixbot-gitlab> gitlab # [ 2.735120] EXT4-fs (vda): mounted filesystem 05d7ae43-c008-4a40-9046-e98c94ddd7fc r/w with ordered data mode. Quota mode: none.634vm-test-run-nixbot-gitlab> gitlab # [ 2.733387] systemd[1]: Mounted /sysroot.635vm-test-run-nixbot-gitlab> gitlab # [ 2.734319] systemd[1]: Reached target Initrd Root File System.636vm-test-run-nixbot-gitlab> gitlab # [ 2.739969] systemd[1]: Mounting /sysroot/nix/.ro-store...637vm-test-run-nixbot-gitlab> gitlab # [ 2.747862] systemd[1]: Mounting /sysroot/nix/.rw-store...638vm-test-run-nixbot-gitlab> gitlab # [ 2.763804] systemd[1]: Mounting /sysroot/run...639vm-test-run-nixbot-gitlab> gitlab # [ 2.769354] systemd[1]: Mounting /sysroot/tmp/shared...640vm-test-run-nixbot-gitlab> gitlab # [ 2.776693] systemd[1]: Mounting /sysroot/tmp/xchg...[ 2.781539] erofs (device vdb): mounted with root inode @ nid 128.641vm-test-run-nixbot-gitlab> gitlab # 642vm-test-run-nixbot-gitlab> gitlab # [ 2.783713] systemd[1]: Starting Mountpoints Configured in the Real Root...643vm-test-run-nixbot-gitlab> gitlab # [ 2.788824] systemd[1]: Mounted /sysroot/nix/.ro-store.644vm-test-run-nixbot-gitlab> gitlab # [ 2.793490] systemd[1]: Mounted /sysroot/nix/.rw-store.645vm-test-run-nixbot-gitlab> gitlab # [ 2.794430] systemd[1]: Mounted /sysroot/run.646vm-test-run-nixbot-gitlab> gitlab # [ 2.795188] systemd[1]: Mounted /sysroot/tmp/shared.647vm-test-run-nixbot-gitlab> gitlab # [ 2.796044] systemd[1]: Mounted /sysroot/tmp/xchg.648vm-test-run-nixbot-gitlab> gitlab # [ 2.800661] systemd-sysroot-fstab-check[259]: /sysroot should be mounted in the initrd, will request daemon-reload.649vm-test-run-nixbot-gitlab> gitlab # [ 2.802326] systemd[1]: Starting rw-sysroot-nix-store.service...650vm-test-run-nixbot-gitlab> gitlab # [ 2.803285] systemd[1]: Reload requested from client PID 259 ('systemd-sysroot') (unit initrd-parse-etc.service)...651vm-test-run-nixbot-gitlab> gitlab # [ 2.805138] systemd[1]: Reloading...652vm-test-run-nixbot-gitlab> gitlab # [ 3.086355] systemd[1]: Reloading finished in 283 ms.653vm-test-run-nixbot-gitlab> gitlab # [ 3.133941] systemd-sysroot-fstab-check[259]: Requesting initrd-fs.target/start/replace...654vm-test-run-nixbot-gitlab> gitlab # [ 3.234018] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.655vm-test-run-nixbot-gitlab> gitlab # [ 3.238458] systemd[1]: Finished rw-sysroot-nix-store.service.656vm-test-run-nixbot-gitlab> gitlab # [ 3.242130] systemd-sysroot-fstab-check[259]: Requesting swap.target/start/replace...657vm-test-run-nixbot-gitlab> gitlab # [ 3.247489] systemd[1]: initrd-parse-etc.service: Deactivated successfully.658vm-test-run-nixbot-gitlab> gitlab # [ 3.250728] systemd[1]: Finished Mountpoints Configured in the Real Root.659vm-test-run-nixbot-gitlab> gitlab # [ 3.252038] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.660vm-test-run-nixbot-gitlab> gitlab # [ 3.256777] (udev-worker)[223]: mtd0ro: Failed to find and pin callout binary "/nix/store/3x4plcsrj2r4mmzis46wrmyka8g4jhi8-systemd-260.1/lib/udev/mtd_probe": No such file or directory661vm-test-run-nixbot-gitlab> gitlab # [ 3.259682] (udev-worker)[223]: mtd0ro: /nix/store/3x4plcsrj2r4mmzis46wrmyka8g4jhi8-systemd-260.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory662vm-test-run-nixbot-gitlab> gitlab # [ 3.264401] systemd[1]: Starting rw-sysroot-nix-store.service...663vm-test-run-nixbot-gitlab> gitlab # [ 3.271653] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.664vm-test-run-nixbot-gitlab> gitlab # [ 3.272799] systemd[1]: Stopped Virtual Console Setup.665vm-test-run-nixbot-gitlab> gitlab # [ 3.274869] systemd[1]: Stopping Virtual Console Setup...666vm-test-run-nixbot-gitlab> gitlab # [ 3.283064] systemd[1]: Starting Virtual Console Setup...667vm-test-run-nixbot-gitlab> gitlab # [ 3.286862] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.668vm-test-run-nixbot-gitlab> gitlab # [ 3.288657] systemd[1]: Finished rw-sysroot-nix-store.service.669vm-test-run-nixbot-gitlab> gitlab # [ 3.303939] systemd-vconsole-setup[488]: Configuration of first virtual console was skipped, ignoring remaining ones.670vm-test-run-nixbot-gitlab> gitlab # [ 3.306923] systemd[1]: Finished Virtual Console Setup.671vm-test-run-nixbot-gitlab> gitlab # [ 3.635375] systemd[1]: Mounting /sysroot/nix/store...672vm-test-run-nixbot-gitlab> gitlab # [ 3.705742] systemd[1]: Mounted /sysroot/nix/store.673vm-test-run-nixbot-gitlab> gitlab # [ 3.708085] systemd[1]: Reached target Initrd File Systems.674vm-test-run-nixbot-gitlab> gitlab # [ 3.713757] systemd[1]: Starting Find NixOS closure...675vm-test-run-nixbot-gitlab> gitlab # [ 3.720883] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...676vm-test-run-nixbot-gitlab> gitlab # [ 3.770456] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.677vm-test-run-nixbot-gitlab> gitlab # [ 3.774238] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.678vm-test-run-nixbot-gitlab> gitlab # [ 3.796800] systemd[1]: Finished Find NixOS closure.679vm-test-run-nixbot-gitlab> gitlab # [ 3.799178] systemd[1]: Reached target Initrd Default Target.680vm-test-run-nixbot-gitlab> gitlab # [ 3.806094] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...681vm-test-run-nixbot-gitlab> gitlab # [ 3.853657] systemd[1]: Stopped target Initrd Default Target.682vm-test-run-nixbot-gitlab> gitlab # [ 3.856384] systemd[1]: Stopped target Basic System.683vm-test-run-nixbot-gitlab> gitlab # [ 3.858641] systemd[1]: Stopped target Initrd Root Device.684vm-test-run-nixbot-gitlab> gitlab # [ 3.861133] systemd[1]: Stopped target Path Units.685vm-test-run-nixbot-gitlab> gitlab # [ 3.863292] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.686vm-test-run-nixbot-gitlab> gitlab # [ 3.866674] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.687vm-test-run-nixbot-gitlab> gitlab # [ 3.870120] systemd[1]: Stopped target Slice Units.688vm-test-run-nixbot-gitlab> gitlab # [ 3.872338] systemd[1]: Stopped target Socket Units.689vm-test-run-nixbot-gitlab> gitlab # [ 3.874713] systemd[1]: Stopped target System Initialization.690vm-test-run-nixbot-gitlab> gitlab # [ 3.877275] systemd[1]: Stopped target Swaps.691vm-test-run-nixbot-gitlab> gitlab # [ 3.879267] systemd[1]: Stopped target Timer Units.692vm-test-run-nixbot-gitlab> gitlab # [ 3.881568] systemd[1]: dbus.socket: Deactivated successfully.693vm-test-run-nixbot-gitlab> gitlab # [ 3.884171] systemd[1]: Closed D-Bus System Message Bus Socket.694vm-test-run-nixbot-gitlab> gitlab # [ 3.886798] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.695vm-test-run-nixbot-gitlab> gitlab # [ 3.890156] systemd[1]: Stopped Find NixOS closure.696vm-test-run-nixbot-gitlab> gitlab # [ 3.892167] systemd[1]: Starting Load Kernel Module 9pnet_virtio...697vm-test-run-nixbot-gitlab> gitlab # [ 3.894569] systemd[1]: Starting rw-sysroot-nix-store.service...698vm-test-run-nixbot-gitlab> gitlab # [ 3.896796] systemd[1]: systemd-sysctl.service: Deactivated successfully.699vm-test-run-nixbot-gitlab> gitlab # [ 3.899209] systemd[1]: Stopped Apply Kernel Variables.700vm-test-run-nixbot-gitlab> gitlab # [ 3.901218] systemd[1]: systemd-modules-load.service: Deactivated successfully.701vm-test-run-nixbot-gitlab> gitlab # [ 3.903627] systemd[1]: Stopped Load Kernel Modules.702vm-test-run-nixbot-gitlab> gitlab # [ 3.910579] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.703vm-test-run-nixbot-gitlab> gitlab # [ 3.914956] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.704vm-test-run-nixbot-gitlab> gitlab # [ 3.920963] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.705vm-test-run-nixbot-gitlab> gitlab # [ 3.924415] systemd[1]: Stopped Create System Files and Directories.706vm-test-run-nixbot-gitlab> gitlab # [ 3.929738] systemd[1]: Stopped target Local File Systems.707vm-test-run-nixbot-gitlab> gitlab # [ 3.932256] systemd[1]: Stopped target Preparation for Local File Systems.708vm-test-run-nixbot-gitlab> gitlab # [ 3.935428] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.709vm-test-run-nixbot-gitlab> gitlab # [ 3.938794] systemd[1]: Stopped Coldplug All udev Devices.710vm-test-run-nixbot-gitlab> gitlab # [ 3.941299] systemd[1]: Stopping Rule-based Manager for Device Events and Files...711vm-test-run-nixbot-gitlab> gitlab # [ 3.944544] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.712vm-test-run-nixbot-gitlab> gitlab # [ 3.947552] systemd[1]: Stopped Virtual Console Setup.713vm-test-run-nixbot-gitlab> gitlab # [ 3.955800] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.714vm-test-run-nixbot-gitlab> gitlab # [ 3.960331] systemd[1]: Finished Load Kernel Module 9pnet_virtio.715vm-test-run-nixbot-gitlab> gitlab # [ 3.970044] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.716vm-test-run-nixbot-gitlab> gitlab # [ 3.974895] systemd[1]: Finished rw-sysroot-nix-store.service.717vm-test-run-nixbot-gitlab> gitlab # [ 3.991761] systemd[1]: systemd-udevd.service: Deactivated successfully.718vm-test-run-nixbot-gitlab> gitlab # [ 3.996951] systemd[1]: Stopped Rule-based Manager for Device Events and Files.719vm-test-run-nixbot-gitlab> gitlab # [ 4.000192] systemd[1]: systemd-udevd.service: Consumed 2.398s CPU time over 2.241s wall clock time, 32.8M memory peak.720vm-test-run-nixbot-gitlab> gitlab # [ 4.007069] systemd[1]: initrd-cleanup.service: Deactivated successfully.721vm-test-run-nixbot-gitlab> gitlab # [ 4.011881] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.722vm-test-run-nixbot-gitlab> gitlab # [ 4.027547] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.723vm-test-run-nixbot-gitlab> gitlab # [ 4.030791] systemd[1]: Closed udev Control Socket.724vm-test-run-nixbot-gitlab> gitlab # [ 4.043571] systemd[1]: Starting Cleanup udev Database...725vm-test-run-nixbot-gitlab> gitlab # [ 4.046133] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.726vm-test-run-nixbot-gitlab> gitlab # [ 4.049683] systemd[1]: Stopped Create Static Device Nodes in /dev.727vm-test-run-nixbot-gitlab> gitlab # [ 4.052429] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.728vm-test-run-nixbot-gitlab> gitlab # [ 4.055927] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.729vm-test-run-nixbot-gitlab> gitlab # [ 4.060664] systemd[1]: kmod-static-nodes.service: Deactivated successfully.730vm-test-run-nixbot-gitlab> gitlab # [ 4.063850] systemd[1]: Stopped Create List of Static Device Nodes.731vm-test-run-nixbot-gitlab> gitlab # [ 4.135573] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.732vm-test-run-nixbot-gitlab> gitlab # [ 4.139121] systemd[1]: Finished Cleanup udev Database.733vm-test-run-nixbot-gitlab> gitlab # [ 4.146070] systemd[1]: Reached target Switch Root.734vm-test-run-nixbot-gitlab> gitlab # [ 4.151589] systemd[1]: Starting NixOS Activation...735vm-test-run-nixbot-gitlab> gitlab # [ 4.330108] initrd-nixos-activation-start[573]: booting system configuration /nix/store/gm1ja6b7ympm67plwx9k8j0agl42j5a4-nixos-system-gitlab-test736vm-test-run-nixbot-gitlab> gitlab # [ 4.392446] initrd-nixos-activation-start[573]: running activation script...737vm-test-run-nixbot-gitlab> gitlab # [ 4.794301] initrd-nixos-activation-start[596]: setting up /etc...738vm-test-run-nixbot-gitlab> gitlab # [ 4.980303] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.739vm-test-run-nixbot-gitlab> gitlab # [ 4.984649] systemd[1]: Finished NixOS Activation.740vm-test-run-nixbot-gitlab> gitlab # [ 5.001292] systemd[1]: Starting Switch Root...741vm-test-run-nixbot-gitlab> gitlab # [ 5.057159] systemd[1]: Switching root.742vm-test-run-nixbot-gitlab> gitlab # [ 5.637649] systemd-journald[154]: Received SIGTERM from PID 1 (systemd).743vm-test-run-nixbot-gitlab> gitlab # [ 6.298086] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)744vm-test-run-nixbot-gitlab> gitlab # [ 6.298259] systemd[1]: Detected virtualization qemu.745vm-test-run-nixbot-gitlab> gitlab # [ 6.298353] systemd[1]: Detected architecture arm64.746vm-test-run-nixbot-gitlab> gitlab # [ 6.298536] systemd[1]: Detected first boot.747vm-test-run-nixbot-gitlab> gitlab # [ 6.300041] systemd[1]: Initializing machine ID from random generator.748vm-test-run-nixbot-gitlab> gitlab # [ 6.555501] systemd[1]: bpf-restrict-fs: LSM BPF program attached749vm-test-run-nixbot-gitlab> gitlab # [ 6.661418] systemd[1]: Applying preset policy.750vm-test-run-nixbot-gitlab> gitlab # [ 6.870560] systemd[1]: Populated /etc with preset unit settings.751vm-test-run-nixbot-gitlab> gitlab # [ 7.022489] systemd[1]: initrd-switch-root.service: Deactivated successfully.752vm-test-run-nixbot-gitlab> gitlab # [ 7.023063] systemd[1]: Stopped initrd-switch-root.service.753vm-test-run-nixbot-gitlab> gitlab # [ 7.024599] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.754vm-test-run-nixbot-gitlab> gitlab # [ 7.028331] systemd[1]: Created slice Slice /system/getty.755vm-test-run-nixbot-gitlab> gitlab # [ 7.034911] systemd[1]: Created slice GitLab DevOps Platform Slice.756vm-test-run-nixbot-gitlab> gitlab # [ 7.038838] systemd[1]: Created slice User and Session Slice.757vm-test-run-nixbot-gitlab> gitlab # [ 7.039356] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.758vm-test-run-nixbot-gitlab> gitlab # [ 7.040117] systemd[1]: Started Forward Password Requests to Wall Directory Watch.759vm-test-run-nixbot-gitlab> gitlab # [ 7.040875] systemd[1]: Expecting device /dev/hvc0...760vm-test-run-nixbot-gitlab> gitlab # [ 7.041604] systemd[1]: Expecting device /dev/ttyAMA0...761vm-test-run-nixbot-gitlab> gitlab # [ 7.042409] systemd[1]: Reached target Local Encrypted Volumes.762vm-test-run-nixbot-gitlab> gitlab # [ 7.043185] systemd[1]: Stopped target initrd-fs.target.763vm-test-run-nixbot-gitlab> gitlab # [ 7.043929] systemd[1]: Stopped target initrd-root-fs.target.764vm-test-run-nixbot-gitlab> gitlab # [ 7.044708] systemd[1]: Stopped target initrd-switch-root.target.765vm-test-run-nixbot-gitlab> gitlab # [ 7.045488] systemd[1]: Reached target Virtual Machines and Containers.766vm-test-run-nixbot-gitlab> gitlab # [ 7.046192] systemd[1]: Reached target Path Units.767vm-test-run-nixbot-gitlab> gitlab # [ 7.047016] systemd[1]: Reached target Remote File Systems.768vm-test-run-nixbot-gitlab> gitlab # [ 7.047700] systemd[1]: Reached target Slice Units.769vm-test-run-nixbot-gitlab> gitlab # [ 7.048403] systemd[1]: Reached target Swaps.770vm-test-run-nixbot-gitlab> gitlab # [ 7.050323] systemd[1]: Listening on Process Core Dump Socket.771vm-test-run-nixbot-gitlab> gitlab # [ 7.051437] systemd[1]: Listening on Credential Encryption/Decryption.772vm-test-run-nixbot-gitlab> gitlab # [ 7.058707] systemd[1]: Starting Journal Log Access Socket...773vm-test-run-nixbot-gitlab> gitlab # [ 7.059825] systemd[1]: Listening on Journal Audit Socket.774vm-test-run-nixbot-gitlab> gitlab # [ 7.060501] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.775vm-test-run-nixbot-gitlab> gitlab # [ 7.060965] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki776vm-test-run-nixbot-gitlab> gitlab # [ 7.061755] systemd[1]: Listening on udev Control Socket.777vm-test-run-nixbot-gitlab> gitlab # [ 7.116327] systemd[1]: Mounting Huge Pages File System...778vm-test-run-nixbot-gitlab> gitlab # [ 7.122528] systemd[1]: Mounting POSIX Message Queue File System...779vm-test-run-nixbot-gitlab> gitlab # [ 7.127230] systemd[1]: Mounting Kernel Debug File System...780vm-test-run-nixbot-gitlab> gitlab # [ 7.137003] systemd[1]: Mounting Kernel Trace File System...781vm-test-run-nixbot-gitlab> gitlab # [ 7.143238] systemd[1]: Starting Create List of Static Device Nodes...782vm-test-run-nixbot-gitlab> gitlab # [ 7.150033] systemd[1]: Starting Load Kernel Module 9pnet_virtio...783vm-test-run-nixbot-gitlab> gitlab # [ 7.153008] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs784vm-test-run-nixbot-gitlab> gitlab # [ 7.169512] systemd[1]: Mounting Kernel Configuration File System...785vm-test-run-nixbot-gitlab> gitlab # [ 7.170766] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm786vm-test-run-nixbot-gitlab> gitlab # [ 7.172955] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore787vm-test-run-nixbot-gitlab> gitlab # [ 7.182867] systemd[1]: Starting Load Kernel Module fuse...788vm-test-run-nixbot-gitlab> gitlab # [ 7.183545] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67789vm-test-run-nixbot-gitlab> gitlab # [ 7.193623] systemd[1]: Starting Journal Service...790vm-test-run-nixbot-gitlab> gitlab # [ 7.197231] systemd[1]: Starting Load Kernel Modules...791vm-test-run-nixbot-gitlab> gitlab # [ 7.202269] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...792vm-test-run-nixbot-gitlab> gitlab # [ 7.216964] systemd[1]: Starting Remount Root and Kernel File Systems...793vm-test-run-nixbot-gitlab> gitlab # [ 7.217476] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki794vm-test-run-nixbot-gitlab> gitlab # [ 7.218470] fuse: init (API version 7.45)795vm-test-run-nixbot-gitlab> gitlab # [ 7.224717] systemd[1]: Starting Coldplug All udev Devices...796vm-test-run-nixbot-gitlab> gitlab # [ 7.232623] systemd[1]: Listening on Journal Log Access Socket.797vm-test-run-nixbot-gitlab> gitlab # [ 7.234961] systemd[1]: Mounted Huge Pages File System.798vm-test-run-nixbot-gitlab> gitlab # [ 7.236414] systemd[1]: Mounted POSIX Message Queue File System.799vm-test-run-nixbot-gitlab> gitlab # [ 7.237547] systemd[1]: Mounted Kernel Debug File System.800vm-test-run-nixbot-gitlab> gitlab # [ 7.240380] systemd[1]: Mounted Kernel Trace File System.801vm-test-run-nixbot-gitlab> gitlab # [ 7.242509] systemd[1]: Finished Create List of Static Device Nodes.802vm-test-run-nixbot-gitlab> gitlab # [ 7.243531] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.803vm-test-run-nixbot-gitlab> gitlab # [ 7.246601] systemd[1]: Finished Load Kernel Module 9pnet_virtio.804vm-test-run-nixbot-gitlab> gitlab # [ 7.247679] systemd[1]: Mounted Kernel Configuration File System.805vm-test-run-nixbot-gitlab> gitlab # [ 7.249534] systemd[1]: modprobe@fuse.service: Deactivated successfully.806vm-test-run-nixbot-gitlab> gitlab # [ 7.253298] systemd[1]: Finished Load Kernel Module fuse.807vm-test-run-nixbot-gitlab> gitlab # [ 7.254169] systemd-journald[805]: Collecting audit messages is enabled.808vm-test-run-nixbot-gitlab> gitlab # [ 7.261266] systemd[1]: Mounting FUSE Control File System...809vm-test-run-nixbot-gitlab> gitlab # [ 7.265089] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...810vm-test-run-nixbot-gitlab> gitlab # [ 7.266020] systemd[1]: Finished Load Kernel Modules.811vm-test-run-nixbot-gitlab> gitlab # [ 7.273224] systemd[1]: Starting Firewall...812vm-test-run-nixbot-gitlab> gitlab # [ 7.273255] EXT4-fs (vda): re-mounted 05d7ae43-c008-4a40-9046-e98c94ddd7fc.813vm-test-run-nixbot-gitlab> gitlab # [ 7.275585] systemd[1]: Queued start job for default target Multi-User System.814vm-test-run-nixbot-gitlab> gitlab # [ 7.277166] systemd[1]: Unnecessary job was removed for /dev/vdb.815vm-test-run-nixbot-gitlab> gitlab # [ 7.278480] systemd[1]: systemd-journald.service: Deactivated successfully.816vm-test-run-nixbot-gitlab> gitlab # [ 7.279988] systemd-modules-load[806]: Using 3 probe threads817vm-test-run-nixbot-gitlab> gitlab # [ 7.285138] systemd[1]: Starting Apply Kernel Variables...818vm-test-run-nixbot-gitlab> gitlab # [ 7.282731] systemd-modules-load[806]: Module 'atkbd' is built in819vm-test-run-nixbot-gitlab> gitlab # [ 7.283928] systemd-modules-load[806]: Module 'loop' is built in820vm-test-run-nixbot-gitlab> gitlab # [ 7.285351] systemd-modules-load[806]: Inserted module 'tls'821vm-test-run-nixbot-gitlab> gitlab # [ 7.286890] systemd-oomd[807]: No swap; memory pressure usage will be degraded[ 7.291613] systemd[1]: Started Journal Service.822vm-test-run-nixbot-gitlab> gitlab # 823vm-test-run-nixbot-gitlab> gitlab # [ 7.295195] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.824vm-test-run-nixbot-gitlab> gitlab # [ 7.298209] systemd[1]: Finished Remount Root and Kernel File Systems.825vm-test-run-nixbot-gitlab> gitlab # [ 7.300615] systemd[1]: Mounted FUSE Control File System.826vm-test-run-nixbot-gitlab> gitlab # [ 7.310432] systemd[1]: Starting Flush Journal to Persistent Storage...827vm-test-run-nixbot-gitlab> gitlab # [ 7.311899] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore828vm-test-run-nixbot-gitlab> gitlab # [ 7.314844] systemd[1]: Starting Load/Save OS Random Seed...829vm-test-run-nixbot-gitlab> gitlab # [ 7.315836] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki830vm-test-run-nixbot-gitlab> gitlab # [ 7.327589] systemd[1]: Finished Apply Kernel Variables.831vm-test-run-nixbot-gitlab> gitlab # [ 7.329342] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.832vm-test-run-nixbot-gitlab> gitlab # [ 7.334263] systemd[1]: Starting Create Static Device Nodes in /dev...833vm-test-run-nixbot-gitlab> gitlab # [ 7.342867] systemd-journald[805]: Received client request to flush runtime journal.834vm-test-run-nixbot-gitlab> gitlab # [ 7.363857] systemd[1]: Finished Load/Save OS Random Seed.835vm-test-run-nixbot-gitlab> gitlab # [ 7.365977] systemd[1]: Reached target First Boot Complete.836vm-test-run-nixbot-gitlab> gitlab # [ 7.367053] systemd[1]: Finished Create Static Device Nodes in /dev.837vm-test-run-nixbot-gitlab> gitlab # [ 7.368244] systemd[1]: Reached target Preparation for Local File Systems.838vm-test-run-nixbot-gitlab> gitlab # [ 7.369212] systemd[1]: Starting Rule-based Manager for Device Events and Files...839vm-test-run-nixbot-gitlab> gitlab # [ 7.370273] systemd[1]: Finished Flush Journal to Persistent Storage.840vm-test-run-nixbot-gitlab> gitlab # [ 7.443307] systemd-udevd[851]: Using default interface naming scheme 'v260'.841vm-test-run-nixbot-gitlab> gitlab # [ 7.491029] systemd[1]: Started Rule-based Manager for Device Events and Files.842vm-test-run-nixbot-gitlab> gitlab # [ 7.677270] systemd[1]: Finished Firewall.843vm-test-run-nixbot-gitlab> gitlab # [ 7.956710] systemd[1]: Finished Coldplug All udev Devices.844vm-test-run-nixbot-gitlab> gitlab # [ 8.024398] systemd[1]: Mounting /run/wrappers...845vm-test-run-nixbot-gitlab> gitlab # [ 8.056731] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs846vm-test-run-nixbot-gitlab> gitlab # [ 8.086433] systemd[1]: Mounted /run/wrappers.847vm-test-run-nixbot-gitlab> gitlab # [ 8.089430] systemd[1]: Reached target Local File Systems.848vm-test-run-nixbot-gitlab> gitlab # [ 8.090876] systemd[1]: Listening on Boot Loader Control Service Socket.849vm-test-run-nixbot-gitlab> gitlab # [ 8.096352] systemd[1]: Starting register-nix-paths.service...850vm-test-run-nixbot-gitlab> gitlab # [ 8.100464] systemd[1]: Starting Create SUID/SGID Wrappers...851vm-test-run-nixbot-gitlab> gitlab # [ 8.102770] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.852vm-test-run-nixbot-gitlab> gitlab # [ 8.125233] systemd[1]: Starting Save Transient machine-id to Disk...853vm-test-run-nixbot-gitlab> gitlab # [ 8.131337] systemd[1]: Starting Create System Files and Directories...854vm-test-run-nixbot-gitlab> gitlab # [ 8.148659] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.855vm-test-run-nixbot-gitlab> gitlab # [ 8.164806] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.856vm-test-run-nixbot-gitlab> gitlab # [ 8.167729] systemd[1]: Finished Save Transient machine-id to Disk.857vm-test-run-nixbot-gitlab> gitlab # [ 8.194012] systemd[1]: Finished Create System Files and Directories.858vm-test-run-nixbot-gitlab> gitlab # [ 8.195079] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped.859vm-test-run-nixbot-gitlab> gitlab # [ 8.198285] systemd[1]: Starting Rebuild Journal Catalog...860vm-test-run-nixbot-gitlab> gitlab # [ 8.200145] systemd[1]: Starting Record System Boot/Shutdown in UTMP...861vm-test-run-nixbot-gitlab> gitlab # [ 8.236482] systemd[1]: Finished Record System Boot/Shutdown in UTMP.862vm-test-run-nixbot-gitlab> gitlab # [ 8.249605] (udev-worker)[899]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.863vm-test-run-nixbot-gitlab> gitlab # [ 8.252059] (udev-worker)[899]: Network interface NamePolicy= disabled on kernel command line.864vm-test-run-nixbot-gitlab> gitlab # [ 8.258419] systemd[1]: Finished Rebuild Journal Catalog.865vm-test-run-nixbot-gitlab> gitlab # [ 8.260460] (udev-worker)[906]: Network interface NamePolicy= disabled on kernel command line.866vm-test-run-nixbot-gitlab> gitlab # [ 8.263403] systemd[1]: Starting Update is Completed...867vm-test-run-nixbot-gitlab> gitlab # [ 8.288576] systemd[1]: Finished Update is Completed.868vm-test-run-nixbot-gitlab> gitlab # [ 8.292813] systemd[1]: Condition check resulted in Virtio network device being skipped.869vm-test-run-nixbot-gitlab> gitlab # [ 8.294485] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore870vm-test-run-nixbot-gitlab> gitlab # [ 8.296383] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.871vm-test-run-nixbot-gitlab> gitlab # [ 8.297904] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67872vm-test-run-nixbot-gitlab> gitlab # [ 8.300327] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore873vm-test-run-nixbot-gitlab> gitlab # [ 8.302189] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki874vm-test-run-nixbot-gitlab> gitlab # [ 8.303804] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki875vm-test-run-nixbot-gitlab> gitlab # [ 8.361008] mousedev: PS/2 mouse device common for all mice876vm-test-run-nixbot-gitlab> gitlab # [ 8.366964] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.877vm-test-run-nixbot-gitlab> gitlab # [ 8.371264] systemd[1]: Finished Create SUID/SGID Wrappers.878vm-test-run-nixbot-gitlab> gitlab # [ 8.372257] systemd[1]: suid-sgid-wrappers.service: Consumed 162ms CPU time over 268ms wall clock time, 6.6M memory peak, 2.2M read from disk.879vm-test-run-nixbot-gitlab> gitlab # [ 8.659358] systemd[1]: Finished register-nix-paths.service.880vm-test-run-nixbot-gitlab> gitlab # [ 8.663570] systemd[1]: Reached target System Initialization.881vm-test-run-nixbot-gitlab> gitlab # [ 8.664599] systemd[1]: Started Discard unused filesystem blocks once a week.882vm-test-run-nixbot-gitlab> gitlab # [ 8.665715] systemd[1]: Started logrotate.timer.883vm-test-run-nixbot-gitlab> gitlab # [ 8.666447] systemd[1]: Started Daily Cleanup of Temporary Directories.884vm-test-run-nixbot-gitlab> gitlab # [ 8.667415] systemd[1]: Reached target Timer Units.885vm-test-run-nixbot-gitlab> gitlab # [ 8.668187] systemd[1]: Listening on D-Bus System Message Bus Socket.886vm-test-run-nixbot-gitlab> gitlab # [ 8.669289] systemd[1]: Listening on Nix Daemon Socket.887vm-test-run-nixbot-gitlab> gitlab # [ 8.677985] systemd[1]: Starting nixbot CI service socket...888vm-test-run-nixbot-gitlab> gitlab # [ 8.679391] systemd[1]: Listening on Hostname Service Socket.889vm-test-run-nixbot-gitlab> gitlab # [ 8.680743] systemd[1]: Listening on nixbot CI service socket.890vm-test-run-nixbot-gitlab> gitlab # [ 8.681691] systemd[1]: Reached target Socket Units.891vm-test-run-nixbot-gitlab> gitlab # [ 8.685571] systemd[1]: Starting D-Bus System Message Bus...892vm-test-run-nixbot-gitlab> gitlab # [ 8.724341] dbus-broker-launch[1055]: Looking up NSS user entry for 'systemd-timesync'...893vm-test-run-nixbot-gitlab> gitlab # [ 8.727084] dbus-broker-launch[1055]: NSS returned no entry for 'systemd-timesync'894vm-test-run-nixbot-gitlab> gitlab # [ 8.728403] dbus-broker-launch[1055]: Invalid user-name in /nix/store/ifm9k575wb3j66gvjx1xkdmmknzqr8v8-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"895vm-test-run-nixbot-gitlab> gitlab # [ 8.732897] systemd[1]: Started D-Bus System Message Bus.896vm-test-run-nixbot-gitlab> gitlab # [ 8.748867] dbus-broker-launch[1055]: Ready897vm-test-run-nixbot-gitlab> gitlab # [ 8.749794] systemd[1]: Reached target Basic System.898vm-test-run-nixbot-gitlab> gitlab # [ 8.752103] systemd[1]: Started backdoor.service.899vm-test-run-nixbot-gitlab> gitlab # [ 8.756327] systemd[1]: Starting gitlab-config.service...900vm-test-run-nixbot-gitlab> gitlab # [ 8.778467] systemd[1]: Starting Import lastlog data into lastlog2 database...901vm-test-run-nixbot-gitlab> gitlab # [ 8.786066] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:08.0/virtio7/input/input3902vm-test-run-nixbot-gitlab> gitlab # [ 8.784580] systemd[1]: Starting Logrotate configuration check...903vm-test-run-nixbot-gitlab> gitlab # [ 8.792154] systemd[1]: Starting Name Service Cache Daemon (nsncd)...904vm-test-run-nixbot-gitlab> gitlab # [ 8.799967] systemd[1]: Starting Post-Boot Actions...905vm-test-run-nixbot-gitlab> gitlab # [ 8.801178] logrotate[1062]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.906vm-test-run-nixbot-gitlab> gitlab # [ 8.804936] logrotate[1062]: reading config file /nix/store/hk6ay5j24mh4h8mpr5rxlxxxp8v61fnj-logrotate.conf907vm-test-run-nixbot-gitlab> gitlab # [ 8.806365] logrotate[1062]: note: 'monthly' overrides previously specified 'weekly'908vm-test-run-nixbot-gitlab> gitlab # connecting to host...909vm-test-run-nixbot-gitlab> gitlab # [ 8.810140] logrotate[1062]: Reading state from file: /var/lib/logrotate.status910vm-test-run-nixbot-gitlab> gitlab # [ 8.811228] logrotate[1062]: state file /var/lib/logrotate.status does not exist911vm-test-run-nixbot-gitlab> gitlab # [ 8.814922] logrotate[1062]: Allocating hash table for state file, size 64 entries912vm-test-run-nixbot-gitlab> gitlab # [ 8.818726] logrotate[1062]: Handling 4 logs913vm-test-run-nixbot-gitlab> gitlab # [ 8.819418] logrotate[1062]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed914vm-test-run-nixbot-gitlab> gitlab # [ 8.826384] logrotate[1062]: considering log /var/log/btmp915vm-test-run-nixbot-gitlab> gitlab: Guest shell says: b'Spawning backdoor root shell...\n'916vm-test-run-nixbot-gitlab> gitlab: connected to guest root shell917vm-test-run-nixbot-gitlab> gitlab: (connecting took 9.29 seconds)918vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for the VM to finish booting, in 9.29 seconds)919vm-test-run-nixbot-gitlab> gitlab # [ 8.829046] logrotate[1062]: Creating new state920vm-test-run-nixbot-gitlab> gitlab # [ 8.829741] logrotate[1062]: Now: 2026-07-01 08:05921vm-test-run-nixbot-gitlab> gitlab # [ 8.830372] logrotate[1062]: Last rotated at 2026-07-01 08:00922vm-test-run-nixbot-gitlab> gitlab # [ 8.831294] logrotate[1062]: log does not need rotating (log has already been rotated)923vm-test-run-nixbot-gitlab> gitlab # [ 8.836125] logrotate[1062]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed924vm-test-run-nixbot-gitlab> gitlab # [ 8.838174] logrotate[1062]: considering log /var/log/wtmp925vm-test-run-nixbot-gitlab> gitlab # [ 8.838942] logrotate[1062]: Creating new state926vm-test-run-nixbot-gitlab> gitlab # [ 8.839520] logrotate[1062]: Now: 2026-07-01 08:05927vm-test-run-nixbot-gitlab> gitlab # [ 8.841301] logrotate[1062]: Last rotated at 2026-07-01 08:00928vm-test-run-nixbot-gitlab> gitlab # [ 8.842093] logrotate[1062]: log does not need rotating (log has already been rotated)929vm-test-run-nixbot-gitlab> gitlab # [ 8.843138] logrotate[1062]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed930vm-test-run-nixbot-gitlab> gitlab # [ 8.848088] logrotate[1062]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1062)931vm-test-run-nixbot-gitlab> gitlab # [ 8.849434] logrotate[1062]: considering log /var/gitlab/state/log/*.log932vm-test-run-nixbot-gitlab> gitlab # [ 8.850369] logrotate[1062]: log /var/gitlab/state/log/*.log does not exist -- skipping933vm-test-run-nixbot-gitlab> gitlab # [ 8.851555] logrotate[1062]: Creating new state934vm-test-run-nixbot-gitlab> gitlab # [ 8.853457] logrotate[1062]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1062)935vm-test-run-nixbot-gitlab> gitlab # [ 8.854756] logrotate[1062]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed936vm-test-run-nixbot-gitlab> gitlab # [ 8.860413] logrotate[1062]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1062)937vm-test-run-nixbot-gitlab> gitlab # [ 8.861581] logrotate[1062]: considering log /var/log/nginx/*.log938vm-test-run-nixbot-gitlab> gitlab # [ 8.862504] logrotate[1062]: log /var/log/nginx/*.log does not exist -- skipping939vm-test-run-nixbot-gitlab> gitlab # [ 8.863530] logrotate[1062]: Creating new state940vm-test-run-nixbot-gitlab> gitlab # [ 8.865351] logrotate[1062]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1062)941vm-test-run-nixbot-gitlab> gitlab # [ 8.866730] systemd[1]: Started Reset console on configuration changes.942vm-test-run-nixbot-gitlab> gitlab # [ 8.871701] systemd[1]: Starting resolvconf update...943vm-test-run-nixbot-gitlab> gitlab # [ 8.872950] nsncd[1065]: Jul 01 08:05:34.728 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"944vm-test-run-nixbot-gitlab> gitlab # [ 8.874959] systemd[1]: Finished Logrotate configuration check.945vm-test-run-nixbot-gitlab> gitlab # [ 8.879707] systemd[1]: Finished Post-Boot Actions.946vm-test-run-nixbot-gitlab> gitlab # [ 8.881782] systemd[1]: Started Name Service Cache Daemon (nsncd).947vm-test-run-nixbot-gitlab> gitlab # [ 8.882862] systemd[1]: Reached target Host and Network Name Lookups.948vm-test-run-nixbot-gitlab> gitlab # [ 8.884038] systemd[1]: Reached target User and Group Name Lookups.949vm-test-run-nixbot-gitlab> gitlab # [ 8.884960] systemd[1]: Starting User Login Management...950vm-test-run-nixbot-gitlab> gitlab # [ 8.899724] systemd[1]: Finished Import lastlog data into lastlog2 database.951vm-test-run-nixbot-gitlab> gitlab # [ 8.936459] systemd[1]: Stopped target Host and Network Name Lookups.952vm-test-run-nixbot-gitlab> gitlab # [ 8.938940] systemd[1]: Stopping Host and Network Name Lookups...953vm-test-run-nixbot-gitlab> gitlab # [ 8.939976] systemd[1]: Stopped target User and Group Name Lookups.954vm-test-run-nixbot-gitlab> gitlab # [ 8.941234] systemd[1]: Stopping User and Group Name Lookups...955vm-test-run-nixbot-gitlab> gitlab # [ 8.942222] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...956vm-test-run-nixbot-gitlab> gitlab # [ 8.943244] systemd[1]: nscd.service: Deactivated successfully.957vm-test-run-nixbot-gitlab> gitlab # [ 8.945589] systemd[1]: Stopped Name Service Cache Daemon (nsncd).958vm-test-run-nixbot-gitlab> gitlab # [ 8.946575] systemd[1]: nscd.service: Consumed 27ms CPU time over 150ms wall clock time, 7.9M memory peak, 5.9M read from disk.959vm-test-run-nixbot-gitlab> gitlab # [ 8.954704] systemd-logind[1107]: New seat seat0.960vm-test-run-nixbot-gitlab> gitlab # [ 8.957248] systemd-logind[1107]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard)961vm-test-run-nixbot-gitlab> gitlab # [ 8.960167] systemd-logind[1107]: Watching system buttons on /dev/input/event0 (gpio-keys)962vm-test-run-nixbot-gitlab> gitlab # [ 8.961338] systemd[1]: Starting Name Service Cache Daemon (nsncd)...963vm-test-run-nixbot-gitlab> gitlab # [ 8.969438] systemd[1]: Started User Login Management.964vm-test-run-nixbot-gitlab> gitlab # [ 8.988002] systemd[1]: Starting linger-users.service...965vm-test-run-nixbot-gitlab> gitlab # [ 8.999305] systemd[1]: Finished resolvconf update.966vm-test-run-nixbot-gitlab> gitlab # [ 9.012530] systemd[1]: Reached target Preparation for Network.967vm-test-run-nixbot-gitlab> gitlab # [ 9.017762] systemd[1]: Starting DHCP Client...968vm-test-run-nixbot-gitlab> gitlab # [ 9.025958] systemd[1]: Starting Address configuration of eth1...969vm-test-run-nixbot-gitlab> gitlab # [ 9.028625] systemd[1]: Starting Extra networking commands....970vm-test-run-nixbot-gitlab> gitlab # [ 9.033099] nsncd[1145]: Jul 01 08:05:34.902 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"971vm-test-run-nixbot-gitlab> gitlab # [ 9.036865] systemd[1]: Starting Virtual Console Setup...972vm-test-run-nixbot-gitlab> gitlab # [ 9.042049] systemd-logind[1107]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)973vm-test-run-nixbot-gitlab> gitlab # [ 9.043358] systemd[1]: Started Name Service Cache Daemon (nsncd).974vm-test-run-nixbot-gitlab> gitlab # [ 9.044861] systemd[1]: linger-users.service: Deactivated successfully.975vm-test-run-nixbot-gitlab> gitlab # [ 9.045922] systemd[1]: Finished linger-users.service.976vm-test-run-nixbot-gitlab> gitlab # [ 9.059922] systemd[1]: Reached target Host and Network Name Lookups.977vm-test-run-nixbot-gitlab> gitlab # [ 9.063287] systemd[1]: Reached target User and Group Name Lookups.978vm-test-run-nixbot-gitlab> gitlab # [ 9.065238] network-addresses-eth1-start[1167]: adding address 192.168.1.1/24... done979vm-test-run-nixbot-gitlab> gitlab # [ 9.068611] network-addresses-eth1-start[1167]: adding address 2001:db8:1::1/64... done980vm-test-run-nixbot-gitlab> gitlab # [ 9.081067] systemd[1]: Finished Address configuration of eth1.981vm-test-run-nixbot-gitlab> gitlab # [ 9.107727] systemd-vconsole-setup[1172]: Configuration of first virtual console was skipped, ignoring remaining ones.982vm-test-run-nixbot-gitlab> gitlab # [ 9.110559] systemd[1]: Finished Virtual Console Setup.983vm-test-run-nixbot-gitlab> gitlab # [ 9.115889] dhcpcd[1185]: dhcpcd-10.3.2 starting984vm-test-run-nixbot-gitlab> gitlab # [ 9.118639] dhcpcd[1232]: dev: loaded udev985vm-test-run-nixbot-gitlab> gitlab # [ 9.128593] systemd[1]: Finished Extra networking commands..986vm-test-run-nixbot-gitlab> gitlab # [ 9.129880] systemd[1]: Reached target Network.[ 9.134715] 8021q: 802.1Q VLAN Support v1.8987vm-test-run-nixbot-gitlab> gitlab # 988vm-test-run-nixbot-gitlab> gitlab # [ 9.135133] 8021q: adding VLAN 0 to HW filter on device eth1989vm-test-run-nixbot-gitlab> gitlab # [ 9.135476] systemd[1]: Starting gitlab-workhorse.service...990vm-test-run-nixbot-gitlab> gitlab # [ 9.140116] systemd[1]: Starting Nginx Web Server...991vm-test-run-nixbot-gitlab> gitlab # [ 9.147155] systemd[1]: Starting PostgreSQL Server...992vm-test-run-nixbot-gitlab> gitlab # [ 9.150085] systemd[1]: Starting Redis Server - redis-gitlab...993vm-test-run-nixbot-gitlab> gitlab # [ 9.156548] systemd[1]: Starting Permit User Sessions...994vm-test-run-nixbot-gitlab> gitlab # [ 9.165521] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.995vm-test-run-nixbot-gitlab> gitlab # [ 9.178339] cfg80211: Loading compiled-in X.509 certificates for regulatory database996vm-test-run-nixbot-gitlab> gitlab # [ 9.182789] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'997vm-test-run-nixbot-gitlab> gitlab # [ 9.183290] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'998vm-test-run-nixbot-gitlab> gitlab # [ 9.184845] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2999vm-test-run-nixbot-gitlab> gitlab # [ 9.185159] cfg80211: failed to load regulatory.db1000vm-test-run-nixbot-gitlab> gitlab # [ 9.210008] systemd[1]: Finished Permit User Sessions.1001vm-test-run-nixbot-gitlab> gitlab # [ 9.214109] systemd[1]: Started Getty on tty1.1002vm-test-run-nixbot-gitlab> gitlab # [ 9.215746] systemd[1]: Reached target Login Prompts.1003vm-test-run-nixbot-gitlab> gitlab # [ 9.224995] 8021q: adding VLAN 0 to HW filter on device eth01004vm-test-run-nixbot-gitlab> gitlab # [ 9.221758] dhcpcd[1232]: eth0: waiting for carrier1005vm-test-run-nixbot-gitlab> gitlab # [ 9.224289] dhcpcd[1232]: eth0: carrier acquired1006vm-test-run-nixbot-gitlab> gitlab # [ 9.227117] dhcpcd[1232]: DUID 00:01:00:01:31:d7:86:cf:52:54:00:12:34:561007vm-test-run-nixbot-gitlab> gitlab # [ 9.228269] dhcpcd[1232]: eth0: IAID 00:12:34:561008vm-test-run-nixbot-gitlab> gitlab # [ 9.229048] dhcpcd[1232]: eth0: adding address fe80::5054:ff:fe12:34561009vm-test-run-nixbot-gitlab> gitlab # [ 9.331286] redis-server[1274]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.1010vm-test-run-nixbot-gitlab> gitlab # [ 9.334310] redis-server[1274]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1011vm-test-run-nixbot-gitlab> gitlab # [ 9.335601] redis-server[1274]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1274, just started1012vm-test-run-nixbot-gitlab> gitlab # [ 9.337482] redis-server[1274]: Configuration loaded1013vm-test-run-nixbot-gitlab> gitlab # [ 9.339833] nginx-pre-start[1275]: nginx: the configuration file /nix/store/2526shpjv0bzachwf4qi968gxiiyv4kq-nginx.conf syntax is ok1014vm-test-run-nixbot-gitlab> gitlab # [ 9.341651] nginx-pre-start[1275]: nginx: configuration file /nix/store/2526shpjv0bzachwf4qi968gxiiyv4kq-nginx.conf test is successful1015vm-test-run-nixbot-gitlab> gitlab # [ 9.343636] systemd[1]: Started Redis Server - redis-gitlab.1016vm-test-run-nixbot-gitlab> gitlab # [ 9.344543] redis[1274]: monotonic clock: ARM CNTVCT @ 1000 ticks/us1017vm-test-run-nixbot-gitlab> gitlab # [ 9.345706] systemd[1]: Started Nginx Web Server.1018vm-test-run-nixbot-gitlab> gitlab # [ 9.346445] redis[1274]: Running mode=standalone, port=0.1019vm-test-run-nixbot-gitlab> gitlab # [ 9.347443] redis[1274]: Server initialized1020vm-test-run-nixbot-gitlab> gitlab # [ 9.348222] redis[1274]: Ready to accept connections unix1021vm-test-run-nixbot-gitlab> gitlab # [ 9.349140] redis[1274]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1022vm-test-run-nixbot-gitlab> gitlab # [ 9.353805] postgresql-pre-start[1277]: The files belonging to this database system will be owned by user "postgres".1023vm-test-run-nixbot-gitlab> gitlab # [ 9.355391] postgresql-pre-start[1277]: This user must also own the server process.1024vm-test-run-nixbot-gitlab> gitlab # [ 9.357789] postgresql-pre-start[1277]: The database cluster will be initialized with locale "en_US.UTF-8".1025vm-test-run-nixbot-gitlab> gitlab # [ 9.359272] postgresql-pre-start[1277]: The default database encoding has accordingly been set to "UTF8".1026vm-test-run-nixbot-gitlab> gitlab # [ 9.361064] postgresql-pre-start[1277]: The default text search configuration will be set to "english".1027vm-test-run-nixbot-gitlab> gitlab # [ 9.362474] postgresql-pre-start[1277]: Data page checksums are disabled.1028vm-test-run-nixbot-gitlab> gitlab # [ 9.363638] postgresql-pre-start[1277]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok1029vm-test-run-nixbot-gitlab> gitlab # [ 9.365373] postgresql-pre-start[1277]: creating subdirectories ... ok1030vm-test-run-nixbot-gitlab> gitlab # [ 9.366335] postgresql-pre-start[1277]: selecting dynamic shared memory implementation ... posix1031vm-test-run-nixbot-gitlab> gitlab # [ 9.397467] postgresql-pre-start[1277]: selecting default "max_connections" ... 1001032vm-test-run-nixbot-gitlab> gitlab # [ 9.457048] postgresql-pre-start[1277]: selecting default "shared_buffers" ... 128MB1033vm-test-run-nixbot-gitlab> gitlab # [ 9.629399] systemd[1]: Started gitlab-workhorse.service.1034vm-test-run-nixbot-gitlab> gitlab # [ 9.876617] postgresql-pre-start[1277]: selecting default time zone ... UTC1035vm-test-run-nixbot-gitlab> gitlab # [ 9.880470] postgresql-pre-start[1277]: creating configuration files ... ok1036vm-test-run-nixbot-gitlab> gitlab # [ 9.999725] gitlab-workhorse[1300]: time="2026-07-01T08:05:35Z" level=info msg=Starting build_time=19700101.000000 version=18.11.41037vm-test-run-nixbot-gitlab> gitlab # [ 10.004175] gitlab-workhorse[1300]: time="2026-07-01T08:05:35Z" level=info msg="Using redis/go-redis"1038vm-test-run-nixbot-gitlab> gitlab # [ 10.005617] gitlab-workhorse[1300]: time="2026-07-01T08:05:35Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1039vm-test-run-nixbot-gitlab> gitlab # [ 10.139897] postgresql-pre-start[1277]: running bootstrap script ... ok1040vm-test-run-nixbot-gitlab> gitlab # [ 10.657263] dhcpcd[1232]: eth0: soliciting a DHCP lease1041vm-test-run-nixbot-gitlab> gitlab # [ 10.677937] dhcpcd[1232]: eth0: offered 10.0.2.15 from 10.0.2.21042vm-test-run-nixbot-gitlab> gitlab # [ 10.696669] dhcpcd[1232]: eth0: probing address 10.0.2.15/241043vm-test-run-nixbot-gitlab> gitlab # [ 10.772727] postgresql-pre-start[1277]: performing post-bootstrap initialization ... ok1044vm-test-run-nixbot-gitlab> gitlab # [ 10.958850] dhcpcd[1232]: eth0: soliciting an IPv6 router1045vm-test-run-nixbot-gitlab> gitlab # [ 10.961960] dhcpcd[1232]: eth0: Router Advertisement from fe80::21046vm-test-run-nixbot-gitlab> gitlab # [ 10.964944] dhcpcd[1232]: eth0: adding address fec0::5054:ff:fe12:3456/641047vm-test-run-nixbot-gitlab> gitlab # [ 10.968344] dhcpcd[1232]: eth0: adding route to fec0::/641048vm-test-run-nixbot-gitlab> gitlab # [ 10.970937] dhcpcd[1232]: eth0: adding default route via fe80::21049vm-test-run-nixbot-gitlab> gitlab # [ 11.092442] postgresql-pre-start[1277]: syncing data to disk ... ok1050vm-test-run-nixbot-gitlab> gitlab # [ 11.095287] postgresql-pre-start[1277]: initdb: warning: enabling "trust" authentication for local connections1051vm-test-run-nixbot-gitlab> gitlab # [ 11.099603] postgresql-pre-start[1277]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1052vm-test-run-nixbot-gitlab> gitlab # [ 11.107055] postgresql-pre-start[1277]: Success. You can now start the database server using:1053vm-test-run-nixbot-gitlab> gitlab # [ 11.111110] postgresql-pre-start[1277]: pg_ctl -D /var/lib/postgresql/17 -l logfile start1054vm-test-run-nixbot-gitlab> gitlab # [ 11.243703] postgres[1338]: [1338] LOG: starting PostgreSQL 17.10 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit1055vm-test-run-nixbot-gitlab> gitlab # [ 11.250148] postgres[1338]: [1338] LOG: listening on IPv6 address "::1", port 54321056vm-test-run-nixbot-gitlab> gitlab # [ 11.254727] postgres[1338]: [1338] LOG: listening on IPv4 address "127.0.0.1", port 54321057vm-test-run-nixbot-gitlab> gitlab # [ 11.258447] postgres[1338]: [1338] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1058vm-test-run-nixbot-gitlab> gitlab # [ 11.263558] postgres[1345]: [1345] LOG: database system was shut down at 2026-07-01 08:05:36 GMT1059vm-test-run-nixbot-gitlab> gitlab # [ 11.267445] postgres[1338]: [1338] LOG: database system is ready to accept connections1060vm-test-run-nixbot-gitlab> gitlab # [ 11.271966] c0sj6d459nhf3wg9pm6ziy71l4581qq8-gitlab-config[1335]: mkdir -p /var/gitlab/state/home/.ssh: OK1061vm-test-run-nixbot-gitlab> gitlab # [ 11.276637] systemd[1]: Started PostgreSQL Server.1062vm-test-run-nixbot-gitlab> gitlab # [ 11.278845] c0sj6d459nhf3wg9pm6ziy71l4581qq8-gitlab-config[1335]: chmod 700 /var/gitlab/state/home/.ssh: OK1063vm-test-run-nixbot-gitlab> gitlab # [ 11.283050] systemd[1]: Starting PostgreSQL Setup Scripts...1064vm-test-run-nixbot-gitlab> gitlab # [ 11.360251] systemd[1]: Finished gitlab-config.service.1065vm-test-run-nixbot-gitlab> gitlab # [ 11.364292] systemd[1]: Started gitaly.service.1066vm-test-run-nixbot-gitlab> gitlab # [ 11.485328] postgresql-setup-start[1369]: CREATE DATABASE1067vm-test-run-nixbot-gitlab> gitlab # [ 11.545049] postgresql-setup-start[1375]: CREATE ROLE1068vm-test-run-nixbot-gitlab> gitlab # [ 11.572201] postgresql-setup-start[1377]: ALTER DATABASE1069vm-test-run-nixbot-gitlab> gitlab # [ 11.630616] postgresql-setup-start[1387]: CREATE ROLE1070vm-test-run-nixbot-gitlab> gitlab # [ 11.636156] systemd[1]: Finished PostgreSQL Setup Scripts.1071vm-test-run-nixbot-gitlab> gitlab # [ 11.638850] systemd[1]: Reached target PostgreSQL.1072vm-test-run-nixbot-gitlab> gitlab # [ 11.644057] systemd[1]: Starting gitlab-db-config.service...1073vm-test-run-nixbot-gitlab> gitlab # [ 11.649337] systemd[1]: Starting gitlab-postgresql.service...1074vm-test-run-nixbot-gitlab> gitlab # [ 11.849992] gitlab-postgresql-start[1398]: CREATE DATABASE1075vm-test-run-nixbot-gitlab> gitlab # [ 11.910312] gitlab-postgresql-start[1404]: CREATE EXTENSION1076vm-test-run-nixbot-gitlab> gitlab # [ 11.975340] gitlab-postgresql-start[1406]: CREATE EXTENSION1077vm-test-run-nixbot-gitlab> gitlab # [ 11.982043] systemd[1]: Finished gitlab-postgresql.service.1078vm-test-run-nixbot-gitlab> gitlab # [ 12.159725] gitaly[1368]: time="2026-07-01T08:05:38.029Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/krygc3bmd623znp2pddh3fv21q5f3nz2-gitaly.toml' and fix the errors that are printed." pid=13681079vm-test-run-nixbot-gitlab> gitlab # [ 12.172264] gitaly[1368]: time="2026-07-01T08:05:38.029Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=13681080vm-test-run-nixbot-gitlab> gitlab # [ 12.174015] gitaly[1368]: time="2026-07-01T08:05:38.029Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=13681081vm-test-run-nixbot-gitlab> gitlab # [ 12.176134] gitaly[1368]: time="2026-07-01T08:05:38.030Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=13681082vm-test-run-nixbot-gitlab> gitlab # [ 12.177853] gitaly[1368]: time="2026-07-01T08:05:38.030Z" level=info msg="Starting Gitaly" pid=1368 version=18.11.41083vm-test-run-nixbot-gitlab> gitlab # [ 12.386159] gitaly[1368]: time="2026-07-01T08:05:38.256Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=225 pid=13681084vm-test-run-nixbot-gitlab> gitlab # [ 12.389579] gitaly[1368]: time="2026-07-01T08:05:38.256Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=13681085vm-test-run-nixbot-gitlab> gitlab # [ 12.392983] gitaly[1368]: time="2026-07-01T08:05:38.256Z" level=info msg="finished initializing command factory" duration_ms=0 pid=13681086vm-test-run-nixbot-gitlab> gitlab # [ 12.394741] gitaly[1368]: time="2026-07-01T08:05:38.256Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1368/git-exec-2628785816.d/git pid=13681087vm-test-run-nixbot-gitlab> gitlab # [ 12.396720] gitaly[1368]: time="2026-07-01T08:05:38.261Z" level=info msg="finished detecting git version" duration_ms=4 pid=13681088vm-test-run-nixbot-gitlab> gitlab # [ 12.398269] gitaly[1368]: time="2026-07-01T08:05:38.261Z" level=info msg="using Git version" pid=1368 version=2.53.g1089vm-test-run-nixbot-gitlab> gitlab # [ 12.399651] gitaly[1368]: time="2026-07-01T08:05:38.261Z" level=info msg="clearing disk cache object folder" pid=1368 storage=default1090vm-test-run-nixbot-gitlab> gitlab # [ 12.401322] gitaly[1368]: time="2026-07-01T08:05:38.262Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache206154821 pid=1368 storage=default1091vm-test-run-nixbot-gitlab> gitlab # [ 12.403742] gitaly[1368]: time="2026-07-01T08:05:38.262Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache206154821 pid=1368 storage=default1092vm-test-run-nixbot-gitlab> gitlab # [ 12.406633] gitaly[1368]: time="2026-07-01T08:05:38.262Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=13681093vm-test-run-nixbot-gitlab> gitlab # [ 12.409571] gitaly[1368]: time="2026-07-01T08:05:38.262Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=13681094vm-test-run-nixbot-gitlab> gitlab # [ 12.412394] gitaly[1368]: time="2026-07-01T08:05:38.262Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache206154821 clear_duration_ms=0 pid=1368 storage=default1095vm-test-run-nixbot-gitlab> gitlab # [ 12.415386] gitaly[1368]: time="2026-07-01T08:05:38.281Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=13681096vm-test-run-nixbot-gitlab> gitlab # [ 12.417636] gitaly[1368]: time="2026-07-01T08:05:38.281Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1368 schema=unix1097vm-test-run-nixbot-gitlab> gitlab # [ 12.420329] gitaly[1368]: time="2026-07-01T08:05:38.281Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1368/sock.d/intern pid=1368 schema=unix1098vm-test-run-nixbot-gitlab> gitlab # [ 12.423131] gitaly[1368]: time="2026-07-01T08:05:38.282Z" level=info msg="finished tempdir cleaner walk" pid=1368 storage=default time_ms=01099vm-test-run-nixbot-gitlab> gitlab # [ 12.425857] gitaly[1368]: time="2026-07-01T08:05:38.282Z" level=info msg="counting repositories" pid=13681100vm-test-run-nixbot-gitlab> gitlab # [ 12.427695] gitaly[1368]: time="2026-07-01T08:05:38.282Z" level=info msg="starting to count repositories" pid=1368 storage_path=/var/gitlab/state/repositories1101vm-test-run-nixbot-gitlab> gitlab # [ 12.430671] gitaly[1368]: time="2026-07-01T08:05:38.282Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1368 storage_path=/var/gitlab/state/repositories1102vm-test-run-nixbot-gitlab> gitlab # [ 12.433829] gitaly[1368]: time="2026-07-01T08:05:38.282Z" level=info msg="completed counting all repositories" counting_duration=0 pid=13681103vm-test-run-nixbot-gitlab> gitlab # [ 12.435502] gitaly[1368]: time="2026-07-01T08:05:38.284Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=13681104vm-test-run-nixbot-gitlab> gitlab # [ 12.437390] gitaly[1368]: time="2026-07-01T08:05:38.284Z" level=info msg="Started Gitaly" duration_ms=254 pid=13681105vm-test-run-nixbot-gitlab> gitlab # [ 12.438887] gitaly[1368]: time="2026-07-01T08:05:38.285Z" level=info msg="maintenance: daily scheduled" pid=1368 scheduled="2026-07-01 12:00:00 +0000 UTC"1106vm-test-run-nixbot-gitlab> gitlab # [ 12.961652] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1394]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1107vm-test-run-nixbot-gitlab> gitlab # [ 12.971169] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1394]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1108vm-test-run-nixbot-gitlab> gitlab # [ 12.984926] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1394]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1109vm-test-run-nixbot-gitlab> gitlab # [ 15.553106] dhcpcd[1232]: eth0: leased 10.0.2.15 for 86400 seconds1110vm-test-run-nixbot-gitlab> gitlab # [ 15.554263] dhcpcd[1232]: eth0: adding route to 10.0.2.0/241111vm-test-run-nixbot-gitlab> gitlab # [ 15.556988] dhcpcd[1232]: eth0: adding default route via 10.0.2.21112vm-test-run-nixbot-gitlab> gitlab # [ 15.680592] systemd[1]: Started DHCP Client.1113vm-test-run-nixbot-gitlab> gitlab # [ 15.681848] systemd[1]: Reached target Network is Online.1114vm-test-run-nixbot-gitlab> gitlab # [ 15.709572] gitaly[1368]: time="2026-07-01T08:05:41.579Z" level=info msg="License database preloaded" duration_ms=3549 pid=13681115vm-test-run-nixbot-gitlab> gitlab # [ 55.636456] postgres[1846]: [1846] ERROR: relation "feature_gates" does not exist at character 5231116vm-test-run-nixbot-gitlab> gitlab # [ 55.637966] postgres[1846]: [1846] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1117vm-test-run-nixbot-gitlab> gitlab # [ 55.639225] postgres[1846]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1118vm-test-run-nixbot-gitlab> gitlab # [ 55.640606] postgres[1846]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1119vm-test-run-nixbot-gitlab> gitlab # [ 55.641822] postgres[1846]: attidentity AS identity,1120vm-test-run-nixbot-gitlab> gitlab # [ 55.642625] postgres[1846]: attgenerated as attgenerated1121vm-test-run-nixbot-gitlab> gitlab # [ 55.643524] postgres[1846]: FROM pg_attribute a1122vm-test-run-nixbot-gitlab> gitlab # [ 55.644246] postgres[1846]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1123vm-test-run-nixbot-gitlab> gitlab # [ 55.645553] postgres[1846]: LEFT JOIN pg_type t ON a.atttypid = t.oid1124vm-test-run-nixbot-gitlab> gitlab # [ 55.646483] postgres[1846]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1125vm-test-run-nixbot-gitlab> gitlab # [ 55.647945] postgres[1846]: WHERE a.attrelid = '"feature_gates"'::regclass1126vm-test-run-nixbot-gitlab> gitlab # [ 55.649045] postgres[1846]: AND a.attnum > 0 AND NOT a.attisdropped1127vm-test-run-nixbot-gitlab> gitlab # [ 55.650039] postgres[1846]: ORDER BY a.attnum1128vm-test-run-nixbot-gitlab> gitlab # [ 55.650705] postgres[1846]:1129vm-test-run-nixbot-gitlab> gitlab # [ 55.984351] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1394]: W, [2026-07-01T08:06:21.853317 #1394] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1130vm-test-run-nixbot-gitlab> gitlab # [ 60.074971] gitlab-workhorse[1300]: time="2026-07-01T08:06:25Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1131vm-test-run-nixbot-gitlab> gitlab # [ 63.839675] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1928]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1132vm-test-run-nixbot-gitlab> gitlab # [ 63.840219] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1928]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1133vm-test-run-nixbot-gitlab> gitlab # [ 70.082253] gitlab-workhorse[1300]: time="2026-07-01T08:06:35Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1134vm-test-run-nixbot-gitlab> gitlab # [ 80.090203] gitlab-workhorse[1300]: time="2026-07-01T08:06:45Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1135vm-test-run-nixbot-gitlab> gitlab # [ 87.593272] postgres[2022]: [2022] ERROR: relation "table_without_models" does not exist1136vm-test-run-nixbot-gitlab> gitlab # [ 87.593849] postgres[2022]: [2022] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1137vm-test-run-nixbot-gitlab> gitlab # [ 90.091798] gitlab-workhorse[1300]: time="2026-07-01T08:06:55Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1138vm-test-run-nixbot-gitlab> gitlab # [ 100.098666] gitlab-workhorse[1300]: time="2026-07-01T08:07:05Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1139vm-test-run-nixbot-gitlab> gitlab # [ 110.106221] gitlab-workhorse[1300]: time="2026-07-01T08:07:15Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1140vm-test-run-nixbot-gitlab> gitlab # [ 117.685905] systemd[1]: Finished gitlab-db-config.service.1141vm-test-run-nixbot-gitlab> gitlab # [ 117.692092] systemd[1]: Started gitlab-sidekiq.service.1142vm-test-run-nixbot-gitlab> gitlab # [ 117.697519] systemd[1]: Starting gitlab.service...1143vm-test-run-nixbot-gitlab> gitlab # [ 118.224493] bundle[2388]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1144vm-test-run-nixbot-gitlab> gitlab # [ 118.230746] bundle[2388]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1145vm-test-run-nixbot-gitlab> gitlab # [ 118.236793] bundle[2388]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1146vm-test-run-nixbot-gitlab> gitlab # [ 120.114146] gitlab-workhorse[1300]: time="2026-07-01T08:07:25Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1147vm-test-run-nixbot-gitlab> gitlab # [ 122.753363] sidekiq-cluster[2387]: {"severity":"INFO","time":"2026-07-01T08:07:28.596Z","message":"Starting cluster with 1 processes"}1148vm-test-run-nixbot-gitlab> gitlab # [ 122.838143] bundle[2388]: {"timestamp":"2026-07-01T08:07:28.674Z","pid":2388,"message":"* Enabling systemd notification integration"}1149vm-test-run-nixbot-gitlab> gitlab # [ 122.843291] bundle[2388]: {"timestamp":"2026-07-01T08:07:28.707Z","pid":2388,"message":"Puma starting in cluster mode..."}1150vm-test-run-nixbot-gitlab> gitlab # [ 122.847881] bundle[2388]: {"timestamp":"2026-07-01T08:07:28.707Z","pid":2388,"message":"* Puma version: 7.2.0 (\"On The Corner\")"}1151vm-test-run-nixbot-gitlab> gitlab # [ 122.852774] bundle[2388]: {"timestamp":"2026-07-01T08:07:28.707Z","pid":2388,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1152vm-test-run-nixbot-gitlab> gitlab # [ 122.858660] bundle[2388]: {"timestamp":"2026-07-01T08:07:28.707Z","pid":2388,"message":"* Min threads: 0"}1153vm-test-run-nixbot-gitlab> gitlab # [ 122.862688] bundle[2388]: {"timestamp":"2026-07-01T08:07:28.707Z","pid":2388,"message":"* Max threads: 4"}1154vm-test-run-nixbot-gitlab> gitlab # [ 122.866748] bundle[2388]: {"timestamp":"2026-07-01T08:07:28.707Z","pid":2388,"message":"* Environment: production"}1155vm-test-run-nixbot-gitlab> gitlab # [ 122.871103] bundle[2388]: {"timestamp":"2026-07-01T08:07:28.707Z","pid":2388,"message":"* Master PID: 2388"}1156vm-test-run-nixbot-gitlab> gitlab # [ 122.875800] bundle[2388]: {"timestamp":"2026-07-01T08:07:28.707Z","pid":2388,"message":"* Workers: 2"}1157vm-test-run-nixbot-gitlab> gitlab # [ 122.880116] bundle[2388]: {"timestamp":"2026-07-01T08:07:28.707Z","pid":2388,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1158vm-test-run-nixbot-gitlab> gitlab # [ 122.888531] bundle[2388]: {"timestamp":"2026-07-01T08:07:28.707Z","pid":2388,"message":"* Preloading application"}1159vm-test-run-nixbot-gitlab> gitlab # [ 130.118618] gitlab-workhorse[1300]: time="2026-07-01T08:07:35Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1160vm-test-run-nixbot-gitlab> gitlab # [ 140.119888] gitlab-workhorse[1300]: time="2026-07-01T08:07:45Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1161vm-test-run-nixbot-gitlab> gitlab # [ 150.130301] gitlab-workhorse[1300]: time="2026-07-01T08:07:55Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1162vm-test-run-nixbot-gitlab> gitlab # [ 159.903299] bundle[2388]: W, [2026-07-01T08:08:05.772274 #2388] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1163vm-test-run-nixbot-gitlab> gitlab # [ 160.140329] gitlab-workhorse[1300]: time="2026-07-01T08:08:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1164vm-test-run-nixbot-gitlab> gitlab # [ 164.382426] sidekiq-cluster[2429]: 2026-07-01T08:08:10.251Z pid=2429 tid=tth INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2429, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>456990720}1165vm-test-run-nixbot-gitlab> gitlab # [ 164.671083] sidekiq-cluster[2429]: W, [2026-07-01T08:08:10.541084 #2429] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1166vm-test-run-nixbot-gitlab> gitlab # [ 164.730487] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:10.600Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1167vm-test-run-nixbot-gitlab> gitlab # [ 168.399798] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.268Z","message":"GitLab reliable fetch activated!"}1168vm-test-run-nixbot-gitlab> gitlab # [ 168.405048] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.269Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1169vm-test-run-nixbot-gitlab> gitlab # [ 168.431951] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.302Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1170vm-test-run-nixbot-gitlab> gitlab # [ 168.439071] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.305Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1171vm-test-run-nixbot-gitlab> gitlab # [ 168.445872] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.308Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1172vm-test-run-nixbot-gitlab> gitlab # [ 168.453461] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.311Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1173vm-test-run-nixbot-gitlab> gitlab # [ 168.460271] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.313Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1174vm-test-run-nixbot-gitlab> gitlab # [ 168.466934] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.315Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1175vm-test-run-nixbot-gitlab> gitlab # [ 168.472409] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.317Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1176vm-test-run-nixbot-gitlab> gitlab # [ 168.477259] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.319Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1177vm-test-run-nixbot-gitlab> gitlab # [ 168.482113] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.321Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"}1178vm-test-run-nixbot-gitlab> gitlab # [ 168.486732] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.323Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1179vm-test-run-nixbot-gitlab> gitlab # [ 168.491480] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.324Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1180vm-test-run-nixbot-gitlab> gitlab # [ 168.495698] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.327Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1181vm-test-run-nixbot-gitlab> gitlab # [ 168.499465] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.329Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1182vm-test-run-nixbot-gitlab> gitlab # [ 168.503217] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.331Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1183vm-test-run-nixbot-gitlab> gitlab # [ 168.506787] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.333Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1184vm-test-run-nixbot-gitlab> gitlab # [ 168.510227] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.336Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"}1185vm-test-run-nixbot-gitlab> gitlab # [ 168.513535] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.338Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1186vm-test-run-nixbot-gitlab> gitlab # [ 168.516512] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.341Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1187vm-test-run-nixbot-gitlab> gitlab # [ 168.519249] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.344Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1188vm-test-run-nixbot-gitlab> gitlab # [ 168.522339] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.346Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1189vm-test-run-nixbot-gitlab> gitlab # [ 168.525182] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.349Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1190vm-test-run-nixbot-gitlab> gitlab # [ 168.528158] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.352Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1191vm-test-run-nixbot-gitlab> gitlab # [ 168.531136] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.354Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1192vm-test-run-nixbot-gitlab> gitlab # [ 168.533788] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.355Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1193vm-test-run-nixbot-gitlab> gitlab # [ 168.536368] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.358Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1194vm-test-run-nixbot-gitlab> gitlab # [ 168.538868] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.360Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1195vm-test-run-nixbot-gitlab> gitlab # [ 168.541298] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.363Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1196vm-test-run-nixbot-gitlab> gitlab # [ 168.543545] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.366Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1197vm-test-run-nixbot-gitlab> gitlab # [ 168.545949] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.368Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1198vm-test-run-nixbot-gitlab> gitlab # [ 168.548250] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.371Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1199vm-test-run-nixbot-gitlab> gitlab # [ 168.550597] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.374Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1200vm-test-run-nixbot-gitlab> gitlab # [ 168.552825] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.377Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1201vm-test-run-nixbot-gitlab> gitlab # [ 168.555104] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.379Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1202vm-test-run-nixbot-gitlab> gitlab # [ 168.557208] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.382Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1203vm-test-run-nixbot-gitlab> gitlab # [ 168.559341] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.384Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1204vm-test-run-nixbot-gitlab> gitlab # [ 168.561554] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.386Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1205vm-test-run-nixbot-gitlab> gitlab # [ 168.563626] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.388Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1206vm-test-run-nixbot-gitlab> gitlab # [ 168.565908] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.391Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1207vm-test-run-nixbot-gitlab> gitlab # [ 168.567960] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.393Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1208vm-test-run-nixbot-gitlab> gitlab # [ 168.570184] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.395Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1209vm-test-run-nixbot-gitlab> gitlab # [ 168.572251] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.397Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1210vm-test-run-nixbot-gitlab> gitlab # [ 168.574423] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.399Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1211vm-test-run-nixbot-gitlab> gitlab # [ 168.576609] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.401Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1212vm-test-run-nixbot-gitlab> gitlab # [ 168.578778] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.403Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1213vm-test-run-nixbot-gitlab> gitlab # [ 168.580930] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.405Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1214vm-test-run-nixbot-gitlab> gitlab # [ 168.583125] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.408Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1215vm-test-run-nixbot-gitlab> gitlab # [ 168.585494] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.410Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1216vm-test-run-nixbot-gitlab> gitlab # [ 168.587645] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.412Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1217vm-test-run-nixbot-gitlab> gitlab # [ 168.589942] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.415Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1218vm-test-run-nixbot-gitlab> gitlab # [ 168.592055] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.417Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1219vm-test-run-nixbot-gitlab> gitlab # [ 168.594343] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.420Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1220vm-test-run-nixbot-gitlab> gitlab # [ 168.596552] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.422Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1221vm-test-run-nixbot-gitlab> gitlab # [ 168.598663] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.464Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1222vm-test-run-nixbot-gitlab> gitlab # [ 168.603521] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.473Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1223vm-test-run-nixbot-gitlab> gitlab # [ 168.606370] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.476Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1224vm-test-run-nixbot-gitlab> gitlab # [ 168.610455] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.480Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1225vm-test-run-nixbot-gitlab> gitlab # [ 168.612718] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.482Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1226vm-test-run-nixbot-gitlab> gitlab # [ 168.616136] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.486Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1227vm-test-run-nixbot-gitlab> gitlab # [ 168.619449] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.489Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1228vm-test-run-nixbot-gitlab> gitlab # [ 168.621861] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.491Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1229vm-test-run-nixbot-gitlab> gitlab # [ 168.624262] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.493Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1230vm-test-run-nixbot-gitlab> gitlab # [ 168.626326] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.496Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1231vm-test-run-nixbot-gitlab> gitlab # [ 168.628431] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.498Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1232vm-test-run-nixbot-gitlab> gitlab # [ 168.631371] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.501Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1233vm-test-run-nixbot-gitlab> gitlab # [ 168.634246] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.504Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1234vm-test-run-nixbot-gitlab> gitlab # [ 168.637241] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.507Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1235vm-test-run-nixbot-gitlab> gitlab # [ 168.640879] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.511Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1236vm-test-run-nixbot-gitlab> gitlab # [ 168.643004] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.512Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1237vm-test-run-nixbot-gitlab> gitlab # [ 168.645536] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.514Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1238vm-test-run-nixbot-gitlab> gitlab # [ 168.648381] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.516Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1239vm-test-run-nixbot-gitlab> gitlab # [ 168.651068] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.518Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1240vm-test-run-nixbot-gitlab> gitlab # [ 168.654004] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.521Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1241vm-test-run-nixbot-gitlab> gitlab # [ 168.656759] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.522Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1242vm-test-run-nixbot-gitlab> gitlab # [ 168.659336] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.524Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1243vm-test-run-nixbot-gitlab> gitlab # [ 168.664820] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.526Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1244vm-test-run-nixbot-gitlab> gitlab # [ 168.667393] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.528Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1245vm-test-run-nixbot-gitlab> gitlab # [ 168.670106] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.531Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1246vm-test-run-nixbot-gitlab> gitlab # [ 168.672535] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.532Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1247vm-test-run-nixbot-gitlab> gitlab # [ 168.675088] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.535Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1248vm-test-run-nixbot-gitlab> gitlab # [ 168.677442] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.537Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1249vm-test-run-nixbot-gitlab> gitlab # [ 168.679534] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.540Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1250vm-test-run-nixbot-gitlab> gitlab # [ 168.682009] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.541Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1251vm-test-run-nixbot-gitlab> gitlab # [ 168.684399] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.543Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1252vm-test-run-nixbot-gitlab> gitlab # [ 168.686727] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.546Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1253vm-test-run-nixbot-gitlab> gitlab # [ 168.689688] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.548Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1254vm-test-run-nixbot-gitlab> gitlab # [ 168.692294] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.551Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1255vm-test-run-nixbot-gitlab> gitlab # [ 168.694645] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.553Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1256vm-test-run-nixbot-gitlab> gitlab # [ 168.697117] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.556Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1257vm-test-run-nixbot-gitlab> gitlab # [ 168.699410] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.559Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1258vm-test-run-nixbot-gitlab> gitlab # [ 168.701873] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.560Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1259vm-test-run-nixbot-gitlab> gitlab # [ 168.704187] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.562Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1260vm-test-run-nixbot-gitlab> gitlab # [ 168.706503] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.565Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1261vm-test-run-nixbot-gitlab> gitlab # [ 168.708831] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.566Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1262vm-test-run-nixbot-gitlab> gitlab # [ 168.711119] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.568Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1263vm-test-run-nixbot-gitlab> gitlab # [ 168.713667] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.571Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1264vm-test-run-nixbot-gitlab> gitlab # [ 168.715648] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.575Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1265vm-test-run-nixbot-gitlab> gitlab # [ 168.718116] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.577Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1266vm-test-run-nixbot-gitlab> gitlab # [ 168.720381] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.579Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1267vm-test-run-nixbot-gitlab> gitlab # [ 168.722569] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.581Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1268vm-test-run-nixbot-gitlab> gitlab # [ 168.724807] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.590Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1269vm-test-run-nixbot-gitlab> gitlab # [ 168.727037] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.592Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1270vm-test-run-nixbot-gitlab> gitlab # [ 168.729345] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.594Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1271vm-test-run-nixbot-gitlab> gitlab # [ 168.731629] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.597Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1272vm-test-run-nixbot-gitlab> gitlab # [ 168.733831] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.599Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1273vm-test-run-nixbot-gitlab> gitlab # [ 168.736045] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.601Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1274vm-test-run-nixbot-gitlab> gitlab # [ 168.738435] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.604Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1275vm-test-run-nixbot-gitlab> gitlab # [ 168.740908] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.605Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1276vm-test-run-nixbot-gitlab> gitlab # [ 168.743058] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.607Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1277vm-test-run-nixbot-gitlab> gitlab # [ 168.745469] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.610Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1278vm-test-run-nixbot-gitlab> gitlab # [ 168.747691] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:14.612Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1279vm-test-run-nixbot-gitlab> gitlab # [ 170.141415] gitlab-workhorse[1300]: time="2026-07-01T08:08:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1280vm-test-run-nixbot-gitlab> gitlab # [ 180.142536] gitlab-workhorse[1300]: time="2026-07-01T08:08:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1281vm-test-run-nixbot-gitlab> gitlab # [ 190.153597] gitlab-workhorse[1300]: time="2026-07-01T08:08:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1282vm-test-run-nixbot-gitlab> gitlab # [ 200.164512] gitlab-workhorse[1300]: time="2026-07-01T08:08:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1283vm-test-run-nixbot-gitlab> gitlab # [ 206.628823] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:52.497Z","message":"Booted Rails 7.2.3 application in production environment"}1284vm-test-run-nixbot-gitlab> gitlab # [ 206.629256] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:52.498Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1285vm-test-run-nixbot-gitlab> gitlab # [ 206.629440] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:52.498Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1286vm-test-run-nixbot-gitlab> gitlab # [ 206.629580] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:52.498Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1287vm-test-run-nixbot-gitlab> gitlab # [ 206.630324] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:52.500Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1288vm-test-run-nixbot-gitlab> gitlab # [ 209.006587] bundle[2388]: {"timestamp":"2026-07-01T08:08:54.875Z","pid":2388,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1289vm-test-run-nixbot-gitlab> gitlab # [ 209.006918] bundle[2388]: {"timestamp":"2026-07-01T08:08:54.875Z","pid":2388,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1290vm-test-run-nixbot-gitlab> gitlab # [ 209.007109] bundle[2388]: {"timestamp":"2026-07-01T08:08:54.875Z","pid":2388,"message":"! #\u003cThread:0x0000f34b1f66a8d8 /nix/store/09wikq0z4z0s8mkafl8f2b6yp0p78frd-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/09wikq0z4z0s8mkafl8f2b6yp0p78frd-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1291vm-test-run-nixbot-gitlab> gitlab # [ 209.007242] bundle[2388]: {"timestamp":"2026-07-01T08:08:54.875Z","pid":2388,"message":"! #\u003cThread:0x0000f34b1f873490 /nix/store/q2bwpwxril8z19gq9b1nnv72drlwhhg8-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/q2bwpwxril8z19gq9b1nnv72drlwhhg8-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1292vm-test-run-nixbot-gitlab> gitlab # [ 209.007378] bundle[2388]: {"timestamp":"2026-07-01T08:08:54.876Z","pid":2388,"message":"Use Ctrl-C to stop"}1293vm-test-run-nixbot-gitlab> gitlab # [ 209.018740] bundle[2388]: {"timestamp":"2026-07-01T08:08:54.876Z","pid":2388,"message":"Sending status to systemd every 1.0 sec"}1294vm-test-run-nixbot-gitlab> gitlab # [ 209.260617] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:55.127Z","message":"Cleaning working queues"}1295vm-test-run-nixbot-gitlab> gitlab # [ 209.262415] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:55.129Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"4720efab111674a2af992a6f","created_at":"2026-07-01T08:07:22.683Z","trace_propagation_headers":{"sentry-trace":"232841bfddc94164980f8e0505453412-9337b8bbff0341c0","baggage":"sentry-trace_id=232841bfddc94164980f8e0505453412,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"759414259128b79d17153dd11001fdac","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-01T08:07:22.694Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-4720efab111674a2af992a6f: start","job_status":"start","queue_duration_s":92.434935,"scheduling_latency_s":92.435}1296vm-test-run-nixbot-gitlab> gitlab # [ 209.526234] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:08:55.395Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"4720efab111674a2af992a6f","created_at":"2026-07-01T08:07:22.683Z","trace_propagation_headers":{"sentry-trace":"232841bfddc94164980f8e0505453412-9337b8bbff0341c0","baggage":"sentry-trace_id=232841bfddc94164980f8e0505453412,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"759414259128b79d17153dd11001fdac","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-01T08:07:22.694Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-4720efab111674a2af992a6f: done: 0.266453 sec","job_status":"done","queue_duration_s":92.434935,"scheduling_latency_s":92.435,"redis_calls":24,"redis_duration_s":0.005884,"redis_read_bytes":912,"redis_write_bytes":2622,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.002752,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001987,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001145,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.126719,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.15594682,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.266453,"completed_at":"2026-07-01T08:08:55.395Z","load_balancing_strategy":"primary","db_duration_s":0.09586,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1297vm-test-run-nixbot-gitlab> gitlab # [ 210.091240] bundle[2388]: {"timestamp":"2026-07-01T08:08:55.960Z","pid":2388,"message":"- Worker 0 (PID: 3212) booted in 1.0s, phase: 0"}1298vm-test-run-nixbot-gitlab> gitlab # [ 210.144749] bundle[2388]: {"timestamp":"2026-07-01T08:08:56.014Z","pid":2388,"message":"- Worker 1 (PID: 3216) booted in 1.04s, phase: 0"}1299vm-test-run-nixbot-gitlab> gitlab # [ 210.152920] systemd[1]: Started gitlab.service.1300vm-test-run-nixbot-gitlab> gitlab # [ 210.157718] systemd[1]: Reached target Common target for all GitLab services..1301vm-test-run-nixbot-gitlab> gitlab # [ 210.163381] systemd[1]: Starting setup-gitlab.service...1302vm-test-run-nixbot-gitlab> gitlab # [ 210.206955] setup-gitlab-start[3256]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1303vm-test-run-nixbot-gitlab> gitlab # [ 213.365924] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:08:59 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 31161304vm-test-run-nixbot-gitlab> gitlab # [ 213.373855] setup-gitlab-start[3284]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token1305vm-test-run-nixbot-gitlab> gitlab # [ 213.380547] setup-gitlab-start[3285]: ++ jq -r .access_token1306vm-test-run-nixbot-gitlab> gitlab # [ 214.616136] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:09:00 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 12151307vm-test-run-nixbot-gitlab> gitlab # [ 214.622073] setup-gitlab-start[3256]: + BEARER=cea197d264c416dae79fcf35f3023b1b4d35e93ae52ee2d457fc35f43b40961a1308vm-test-run-nixbot-gitlab> gitlab # [ 214.627729] setup-gitlab-start[3256]: + AUTH='Authorization: Bearer cea197d264c416dae79fcf35f3023b1b4d35e93ae52ee2d457fc35f43b40961a'1309vm-test-run-nixbot-gitlab> gitlab # [ 214.632753] setup-gitlab-start[3256]: + curl -fs -X PUT -H 'Authorization: Bearer cea197d264c416dae79fcf35f3023b1b4d35e93ae52ee2d457fc35f43b40961a' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true'1310vm-test-run-nixbot-gitlab> gitlab # [ 215.589722] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:09:01 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 9531311vm-test-run-nixbot-gitlab> gitlab # [ 215.593415] setup-gitlab-start[3256]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear1312vm-test-run-nixbot-gitlab> gitlab # [ 215.608747] runuser[3303]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1313vm-test-run-nixbot-gitlab> gitlab # [ 216.076484] setup-gitlab-start[3304]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1314vm-test-run-nixbot-gitlab> gitlab # [ 216.083009] setup-gitlab-start[3304]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1315vm-test-run-nixbot-gitlab> gitlab # [ 216.089500] setup-gitlab-start[3304]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1316vm-test-run-nixbot-gitlab> gitlab # [ 216.832594] setup-gitlab-start[3304]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1317vm-test-run-nixbot-gitlab> gitlab # [ 216.832929] setup-gitlab-start[3304]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1318vm-test-run-nixbot-gitlab> gitlab # [ 216.837875] setup-gitlab-start[3304]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1319vm-test-run-nixbot-gitlab> gitlab # [ 221.731873] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:07.599Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"907cbe3028896a89fa80829b","created_at":"2026-07-01T08:09:07.558Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"38bfd801fdcef891a09d0be8f84ee0c3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-01T08:09:07.596Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-907cbe3028896a89fa80829b: start","job_status":"start","queue_duration_s":0.002759,"scheduling_latency_s":0.002804}1320vm-test-run-nixbot-gitlab> gitlab # [ 221.884882] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:07.754Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"907cbe3028896a89fa80829b","created_at":"2026-07-01T08:09:07.558Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"38bfd801fdcef891a09d0be8f84ee0c3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-01T08:09:07.596Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-907cbe3028896a89fa80829b: done: 0.155115 sec","job_status":"done","queue_duration_s":0.002759,"scheduling_latency_s":0.002804,"redis_calls":10,"redis_duration_s":0.026916000000000002,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002061,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.021812,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003043,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00079276,"duration_s":0.155115,"completed_at":"2026-07-01T08:09:07.754Z","load_balancing_strategy":"primary","db_duration_s":0.01826,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1321vm-test-run-nixbot-gitlab> gitlab # [ 221.934026] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:07.803Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"9165b3084c489bb87b8d12d5","created_at":"2026-07-01T08:09:07.645Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"981ae9d772a92f41b175c913455bd8a0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:07.680Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-9165b3084c489bb87b8d12d5: start","job_status":"start","queue_duration_s":0.123257,"scheduling_latency_s":0.123283}1322vm-test-run-nixbot-gitlab> gitlab # [ 222.740228] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:08.608Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"9165b3084c489bb87b8d12d5","created_at":"2026-07-01T08:09:07.645Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"981ae9d772a92f41b175c913455bd8a0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:07.680Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-9165b3084c489bb87b8d12d5: done: 0.804153 sec","job_status":"done","queue_duration_s":0.123257,"scheduling_latency_s":0.123283,"redis_calls":14,"redis_duration_s":0.025746,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.010396,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011538,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003812,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.025,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.381079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00307618,"exclusive_lock_hold_duration_s":0.7099998,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.804153,"completed_at":"2026-07-01T08:09:08.607Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.038613,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1323vm-test-run-nixbot-gitlab> gitlab # [ 222.743890] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:08.610Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"e3c1a617a18258a265bf9316","created_at":"2026-07-01T08:09:07.731Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17e71a943f01189431fdae4299f8e024","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:07.777Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-e3c1a617a18258a265bf9316: start","job_status":"start","queue_duration_s":0.833242,"scheduling_latency_s":0.833278}1324vm-test-run-nixbot-gitlab> gitlab # [ 222.864218] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:08.734Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1325vm-test-run-nixbot-gitlab> gitlab # [ 222.882655] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:08.751Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"e3c1a617a18258a265bf9316","created_at":"2026-07-01T08:09:07.731Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17e71a943f01189431fdae4299f8e024","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:07.777Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-e3c1a617a18258a265bf9316: done: 0.14089 sec","job_status":"done","queue_duration_s":0.833242,"scheduling_latency_s":0.833278,"redis_calls":10,"redis_duration_s":0.0038599999999999997,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000696,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002375,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000789,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0006618,"duration_s":0.14089,"completed_at":"2026-07-01T08:09:08.751Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000874,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1326vm-test-run-nixbot-gitlab> gitlab # [ 222.944698] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:08.754Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"8c1bd8ebaca6acf803d8200f","created_at":"2026-07-01T08:09:07.789Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3f61c972682dcf7340b68d92add09c38","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:07.801Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-8c1bd8ebaca6acf803d8200f: start","job_status":"start","queue_duration_s":0.95368,"scheduling_latency_s":0.953722}1327vm-test-run-nixbot-gitlab> gitlab # [ 222.960152] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:08.820Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"8c1bd8ebaca6acf803d8200f","created_at":"2026-07-01T08:09:07.789Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3f61c972682dcf7340b68d92add09c38","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:07.801Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-8c1bd8ebaca6acf803d8200f: done: 0.066061 sec","job_status":"done","queue_duration_s":0.95368,"scheduling_latency_s":0.953722,"redis_calls":12,"redis_duration_s":0.00372,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000614,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001752,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001354,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036432,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0010861,"exclusive_lock_hold_duration_s":0.01393086,"duration_s":0.066061,"completed_at":"2026-07-01T08:09:08.820Z","load_balancing_strategy":"primary","db_duration_s":0.010458,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1328vm-test-run-nixbot-gitlab> gitlab # [ 223.003830] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:08.823Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"9c6b2737d2be957948a5a63a","created_at":"2026-07-01T08:09:07.828Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c74088e6fa67c2841d86c951ac4cd2c","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:07.843Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-9c6b2737d2be957948a5a63a: start","job_status":"start","queue_duration_s":0.979253,"scheduling_latency_s":0.97928}1329vm-test-run-nixbot-gitlab> gitlab # [ 223.018395] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:08.870Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"9c6b2737d2be957948a5a63a","created_at":"2026-07-01T08:09:07.828Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c74088e6fa67c2841d86c951ac4cd2c","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:07.843Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-9c6b2737d2be957948a5a63a: done: 0.047706 sec","job_status":"done","queue_duration_s":0.979253,"scheduling_latency_s":0.97928,"redis_calls":16,"redis_duration_s":0.002812,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001408,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001005,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000399,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029261,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002686,"duration_s":0.047706,"completed_at":"2026-07-01T08:09:08.870Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009485,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1330vm-test-run-nixbot-gitlab> gitlab # [ 223.060759] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:08.872Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"f69875d51ef3b5294004e517","created_at":"2026-07-01T08:09:07.863Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4443483d44a1528d5fde9ae1a4a15676","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:07.882Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-f69875d51ef3b5294004e517: start","job_status":"start","queue_duration_s":0.990039,"scheduling_latency_s":0.990057}1331vm-test-run-nixbot-gitlab> gitlab # [ 223.229405] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.099Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"f69875d51ef3b5294004e517","created_at":"2026-07-01T08:09:07.863Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4443483d44a1528d5fde9ae1a4a15676","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:07.882Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-f69875d51ef3b5294004e517: done: 0.227037 sec","job_status":"done","queue_duration_s":0.990039,"scheduling_latency_s":0.990057,"redis_calls":12,"redis_duration_s":0.002604,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000281,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001352,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":124,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":124,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.03,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.186329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00046374,"exclusive_lock_hold_duration_s":0.20625655999999998,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.227037,"completed_at":"2026-07-01T08:09:09.099Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.046743,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1332vm-test-run-nixbot-gitlab> gitlab # [ 223.321594] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.100Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"afa312d0e9465e83faa3a53d","created_at":"2026-07-01T08:09:07.987Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82c8982a59957ae7fb400b6110e76a38","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.002Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-afa312d0e9465e83faa3a53d: start","job_status":"start","queue_duration_s":1.098097,"scheduling_latency_s":1.098115}1333vm-test-run-nixbot-gitlab> gitlab # [ 223.579929] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.449Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"afa312d0e9465e83faa3a53d","created_at":"2026-07-01T08:09:07.987Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82c8982a59957ae7fb400b6110e76a38","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.002Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-afa312d0e9465e83faa3a53d: done: 0.348684 sec","job_status":"done","queue_duration_s":1.098097,"scheduling_latency_s":1.098115,"redis_calls":574,"redis_duration_s":0.088621,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000499,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001765,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.086357,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.296417,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033878,"duration_s":0.348684,"completed_at":"2026-07-01T08:09:09.449Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000335,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1334vm-test-run-nixbot-gitlab> gitlab # [ 223.669981] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.451Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"18b76468915584a81b13a3c9","created_at":"2026-07-01T08:09:08.018Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cba58cb5a01686fafda75c6ae075941a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.031Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-18b76468915584a81b13a3c9: start","job_status":"start","queue_duration_s":1.419349,"scheduling_latency_s":1.419369}1335vm-test-run-nixbot-gitlab> gitlab # [ 223.687254] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.503Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"18b76468915584a81b13a3c9","created_at":"2026-07-01T08:09:08.018Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cba58cb5a01686fafda75c6ae075941a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.031Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-18b76468915584a81b13a3c9: done: 0.052551 sec","job_status":"done","queue_duration_s":1.419349,"scheduling_latency_s":1.419369,"redis_calls":10,"redis_duration_s":0.002835,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000543,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001673,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000619,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016183,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00051834,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.052551,"completed_at":"2026-07-01T08:09:09.503Z","load_balancing_strategy":"primary","db_duration_s":0.027845,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1336vm-test-run-nixbot-gitlab> gitlab # [ 223.731612] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.505Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ad5c69f298693a1fa7b04b95","created_at":"2026-07-01T08:09:08.044Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e46d97c4c2d740f39a6473a8ee9ae172","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.055Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ad5c69f298693a1fa7b04b95: start","job_status":"start","queue_duration_s":1.449643,"scheduling_latency_s":1.449663}1337vm-test-run-nixbot-gitlab> gitlab # [ 223.746593] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.537Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ad5c69f298693a1fa7b04b95","created_at":"2026-07-01T08:09:08.044Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e46d97c4c2d740f39a6473a8ee9ae172","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.055Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ad5c69f298693a1fa7b04b95: done: 0.031726 sec","job_status":"done","queue_duration_s":1.449643,"scheduling_latency_s":1.449663,"redis_calls":12,"redis_duration_s":0.003404,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000593,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001529,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001282,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00060828,"exclusive_lock_hold_duration_s":0.00808972,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.031726,"completed_at":"2026-07-01T08:09:09.537Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005449,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1338vm-test-run-nixbot-gitlab> gitlab # [ 223.791521] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.539Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"5ddd5352909af6cab365d652","created_at":"2026-07-01T08:09:08.064Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc82975b113d6a89c3f2ed997b3c72cc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.074Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-5ddd5352909af6cab365d652: start","job_status":"start","queue_duration_s":1.46505,"scheduling_latency_s":1.465071}1339vm-test-run-nixbot-gitlab> gitlab # [ 223.806467] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.552Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1340vm-test-run-nixbot-gitlab> gitlab # [ 223.809782] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.560Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"5ddd5352909af6cab365d652","created_at":"2026-07-01T08:09:08.064Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc82975b113d6a89c3f2ed997b3c72cc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.074Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-5ddd5352909af6cab365d652: done: 0.021044 sec","job_status":"done","queue_duration_s":1.46505,"scheduling_latency_s":1.465071,"redis_calls":10,"redis_duration_s":0.0025120000000000003,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00055,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001445,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000517,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033598,"duration_s":0.021044,"completed_at":"2026-07-01T08:09:09.560Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000686,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1341vm-test-run-nixbot-gitlab> gitlab # [ 223.852744] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.561Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"9997b3b4b2788998b9fdfadf","created_at":"2026-07-01T08:09:08.081Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53eea716780d7f95bb66816e4c629477","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.091Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-9997b3b4b2788998b9fdfadf: start","job_status":"start","queue_duration_s":1.470394,"scheduling_latency_s":1.470412}1342vm-test-run-nixbot-gitlab> gitlab # [ 223.867939] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.598Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"9997b3b4b2788998b9fdfadf","created_at":"2026-07-01T08:09:08.081Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53eea716780d7f95bb66816e4c629477","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.091Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-9997b3b4b2788998b9fdfadf: done: 0.036272 sec","job_status":"done","queue_duration_s":1.470394,"scheduling_latency_s":1.470412,"redis_calls":13,"redis_duration_s":0.003483,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000565,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001762,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001156,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021518,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01334284,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.036272,"completed_at":"2026-07-01T08:09:09.598Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006639,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1343vm-test-run-nixbot-gitlab> gitlab # [ 223.912882] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.599Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"b4d8c8af93fa8c499d56a7c6","created_at":"2026-07-01T08:09:08.132Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"684ce3006e04c717e3cb16afa60610ab","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.140Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-b4d8c8af93fa8c499d56a7c6: start","job_status":"start","queue_duration_s":1.458955,"scheduling_latency_s":1.458973}1344vm-test-run-nixbot-gitlab> gitlab # [ 223.927857] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.622Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"b4d8c8af93fa8c499d56a7c6","created_at":"2026-07-01T08:09:08.132Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"684ce3006e04c717e3cb16afa60610ab","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.140Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-b4d8c8af93fa8c499d56a7c6: done: 0.023008 sec","job_status":"done","queue_duration_s":1.458955,"scheduling_latency_s":1.458973,"redis_calls":12,"redis_duration_s":0.002995,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001037,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00147,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000488,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033114,"duration_s":0.023008,"completed_at":"2026-07-01T08:09:09.622Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001378,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1345vm-test-run-nixbot-gitlab> gitlab # [ 223.971581] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.624Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"70f0c3b39a6d967da0204069","created_at":"2026-07-01T08:09:08.153Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b7bf21fc469546465c8806a01d71a21","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.161Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-70f0c3b39a6d967da0204069: start","job_status":"start","queue_duration_s":1.462465,"scheduling_latency_s":1.462482}1346vm-test-run-nixbot-gitlab> gitlab # [ 223.986210] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.636Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1347vm-test-run-nixbot-gitlab> gitlab # [ 223.989812] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.643Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"70f0c3b39a6d967da0204069","created_at":"2026-07-01T08:09:08.153Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b7bf21fc469546465c8806a01d71a21","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.161Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-70f0c3b39a6d967da0204069: done: 0.019473 sec","job_status":"done","queue_duration_s":1.462465,"scheduling_latency_s":1.462482,"redis_calls":10,"redis_duration_s":0.002386,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00049,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001379,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000517,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032346,"duration_s":0.019473,"completed_at":"2026-07-01T08:09:09.643Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000623,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1348vm-test-run-nixbot-gitlab> gitlab # [ 224.032442] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.645Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"29937fc77a51370ecd72edbe","created_at":"2026-07-01T08:09:08.172Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dcdf0b9f22a73081600e8a7081b018f6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.180Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-29937fc77a51370ecd72edbe: start","job_status":"start","queue_duration_s":1.464816,"scheduling_latency_s":1.464832}1349vm-test-run-nixbot-gitlab> gitlab # [ 224.047256] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.657Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1350vm-test-run-nixbot-gitlab> gitlab # [ 224.051216] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.664Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"29937fc77a51370ecd72edbe","created_at":"2026-07-01T08:09:08.172Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dcdf0b9f22a73081600e8a7081b018f6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.180Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-29937fc77a51370ecd72edbe: done: 0.019663 sec","job_status":"done","queue_duration_s":1.464816,"scheduling_latency_s":1.464832,"redis_calls":10,"redis_duration_s":0.0024289999999999997,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000493,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001405,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000531,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033164,"duration_s":0.019663,"completed_at":"2026-07-01T08:09:09.664Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000711,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1351vm-test-run-nixbot-gitlab> gitlab # [ 224.094141] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.666Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"253ab758a0ef937cf52cd93e","created_at":"2026-07-01T08:09:08.192Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.198Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-253ab758a0ef937cf52cd93e: start","job_status":"start","queue_duration_s":1.467514,"scheduling_latency_s":1.467532}1352vm-test-run-nixbot-gitlab> gitlab # [ 224.107772] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.814Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"253ab758a0ef937cf52cd93e","created_at":"2026-07-01T08:09:08.192Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.198Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-253ab758a0ef937cf52cd93e: done: 0.148154 sec","job_status":"done","queue_duration_s":1.467514,"scheduling_latency_s":1.467532,"redis_calls":71,"redis_duration_s":0.020879,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002442,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.0006,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.017229,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000608,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.102247,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023496,"duration_s":0.148154,"completed_at":"2026-07-01T08:09:09.814Z","load_balancing_strategy":"primary","db_duration_s":0.025102,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1353vm-test-run-nixbot-gitlab> gitlab # [ 224.150979] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.815Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"2c287ef1d2356a10149e65a6","created_at":"2026-07-01T08:09:08.205Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a1cc81e6a458c4dc18f0f2337449a2e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.212Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-2c287ef1d2356a10149e65a6: start","job_status":"start","queue_duration_s":1.603125,"scheduling_latency_s":1.603143}1354vm-test-run-nixbot-gitlab> gitlab # [ 224.165428] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.841Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"2c287ef1d2356a10149e65a6","created_at":"2026-07-01T08:09:08.205Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a1cc81e6a458c4dc18f0f2337449a2e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.212Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-2c287ef1d2356a10149e65a6: done: 0.025236 sec","job_status":"done","queue_duration_s":1.603125,"scheduling_latency_s":1.603143,"redis_calls":10,"redis_duration_s":0.001361,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000271,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000799,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000291,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014185,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000237,"duration_s":0.025236,"completed_at":"2026-07-01T08:09:09.841Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004765,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355vm-test-run-nixbot-gitlab> gitlab # [ 224.208571] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.842Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"893eaebabd23f09b0ff6d57d","created_at":"2026-07-01T08:09:08.225Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0bd678529797c36fe7bf066dd95d115b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.231Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-893eaebabd23f09b0ff6d57d: start","job_status":"start","queue_duration_s":1.611275,"scheduling_latency_s":1.611292}1356vm-test-run-nixbot-gitlab> gitlab # [ 224.223397] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.896Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"893eaebabd23f09b0ff6d57d","created_at":"2026-07-01T08:09:08.225Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0bd678529797c36fe7bf066dd95d115b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.231Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-893eaebabd23f09b0ff6d57d: done: 0.053675 sec","job_status":"done","queue_duration_s":1.611275,"scheduling_latency_s":1.611292,"redis_calls":12,"redis_duration_s":0.001969,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000294,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000957,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000718,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000543,"exclusive_lock_hold_duration_s":0.02457486,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.053675,"completed_at":"2026-07-01T08:09:09.896Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026306,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1357vm-test-run-nixbot-gitlab> gitlab # [ 224.268474] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.897Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"629285449c120dc8ffc39510","created_at":"2026-07-01T08:09:08.240Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17e4393d76357921f2234840f18fc5f7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-01T08:09:08.246Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-629285449c120dc8ffc39510: start","job_status":"start","queue_duration_s":1.65105,"scheduling_latency_s":1.651069}1358vm-test-run-nixbot-gitlab> gitlab # [ 224.281647] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.910Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1359vm-test-run-nixbot-gitlab> gitlab # [ 224.285724] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.918Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"629285449c120dc8ffc39510","created_at":"2026-07-01T08:09:08.240Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17e4393d76357921f2234840f18fc5f7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-01T08:09:08.246Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-629285449c120dc8ffc39510: done: 0.020831 sec","job_status":"done","queue_duration_s":1.65105,"scheduling_latency_s":1.651069,"redis_calls":10,"redis_duration_s":0.00148,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000345,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000816,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000319,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012832,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00046992,"duration_s":0.020831,"completed_at":"2026-07-01T08:09:09.918Z","load_balancing_strategy":"primary","db_duration_s":0.000657,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360vm-test-run-nixbot-gitlab> gitlab # [ 224.327598] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.919Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6aaef36efff3ba4ecdd61618","created_at":"2026-07-01T08:09:08.254Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"15130ac15f62021bd0b53eeed9d76ab3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-01T08:09:08.261Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6aaef36efff3ba4ecdd61618: start","job_status":"start","queue_duration_s":1.65854,"scheduling_latency_s":1.658558}1361vm-test-run-nixbot-gitlab> gitlab # [ 224.341350] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.931Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1362vm-test-run-nixbot-gitlab> gitlab # [ 224.344681] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.938Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6aaef36efff3ba4ecdd61618","created_at":"2026-07-01T08:09:08.254Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"15130ac15f62021bd0b53eeed9d76ab3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-01T08:09:08.261Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6aaef36efff3ba4ecdd61618: done: 0.018665 sec","job_status":"done","queue_duration_s":1.65854,"scheduling_latency_s":1.658558,"redis_calls":10,"redis_duration_s":0.001325,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000292,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000747,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000286,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022974,"duration_s":0.018665,"completed_at":"2026-07-01T08:09:09.938Z","load_balancing_strategy":"primary","db_duration_s":0.000542,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1363vm-test-run-nixbot-gitlab> gitlab # [ 224.387690] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.941Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"df4960fc5b6df095bc1e4b72","created_at":"2026-07-01T08:09:08.265Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf4fe4e532e1b7b69c7983ac0e48dcda","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.271Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-df4960fc5b6df095bc1e4b72: start","job_status":"start","queue_duration_s":1.670014,"scheduling_latency_s":1.670034}1364vm-test-run-nixbot-gitlab> gitlab # [ 224.401610] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.976Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"df4960fc5b6df095bc1e4b72","created_at":"2026-07-01T08:09:08.265Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf4fe4e532e1b7b69c7983ac0e48dcda","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.271Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-df4960fc5b6df095bc1e4b72: done: 0.034673 sec","job_status":"done","queue_duration_s":1.670014,"scheduling_latency_s":1.670034,"redis_calls":19,"redis_duration_s":0.0026639999999999997,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000552,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000187,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.001486,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022722,"duration_s":0.034673,"completed_at":"2026-07-01T08:09:09.976Z","load_balancing_strategy":"primary","db_duration_s":0.002966,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1365vm-test-run-nixbot-gitlab> gitlab # [ 224.445015] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.977Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"8b1750bb2d4c78c15e85e79f","created_at":"2026-07-01T08:09:08.279Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8f19cce2accfc3a8b39147ba1666674","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.284Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-8b1750bb2d4c78c15e85e79f: start","job_status":"start","queue_duration_s":1.692335,"scheduling_latency_s":1.692351}1366vm-test-run-nixbot-gitlab> gitlab # [ 224.459963] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.996Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"8b1750bb2d4c78c15e85e79f","created_at":"2026-07-01T08:09:08.279Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8f19cce2accfc3a8b39147ba1666674","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-01T08:09:08.284Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-8b1750bb2d4c78c15e85e79f: done: 0.0192 sec","job_status":"done","queue_duration_s":1.692335,"scheduling_latency_s":1.692351,"redis_calls":10,"redis_duration_s":0.001371,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000293,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000767,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000311,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011484,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002306,"duration_s":0.0192,"completed_at":"2026-07-01T08:09:09.996Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000679,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1367vm-test-run-nixbot-gitlab> gitlab # [ 224.503348] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:09.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74b0086a49bc92177fbc1bad","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74b0086a49bc92177fbc1bad: start","job_status":"start","queue_duration_s":0.191236,"scheduling_latency_s":0.191253}1368vm-test-run-nixbot-gitlab> gitlab # [ 224.516108] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.019Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74b0086a49bc92177fbc1bad","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74b0086a49bc92177fbc1bad: done: 0.021748 sec","job_status":"done","queue_duration_s":0.191236,"scheduling_latency_s":0.191253,"redis_calls":16,"redis_duration_s":0.002118,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000277,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000591,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00125,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013671,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026336,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021748,"completed_at":"2026-07-01T08:09:10.019Z","load_balancing_strategy":"primary","db_duration_s":0.000552,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1369vm-test-run-nixbot-gitlab> gitlab # [ 224.557571] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f7715b46cce004c274c2530","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f7715b46cce004c274c2530: start","job_status":"start","queue_duration_s":0.214218,"scheduling_latency_s":0.214236}1370vm-test-run-nixbot-gitlab> gitlab # [ 224.570214] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.035Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f7715b46cce004c274c2530","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f7715b46cce004c274c2530: done: 0.014962 sec","job_status":"done","queue_duration_s":0.214218,"scheduling_latency_s":0.214236,"redis_calls":13,"redis_duration_s":0.0017239999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000983,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010518,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014962,"completed_at":"2026-07-01T08:09:10.035Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1371vm-test-run-nixbot-gitlab> gitlab # [ 224.611003] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e311c506c7592695f88d8ce","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e311c506c7592695f88d8ce: start","job_status":"start","queue_duration_s":0.230253,"scheduling_latency_s":0.230271}1372vm-test-run-nixbot-gitlab> gitlab # [ 224.623642] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e311c506c7592695f88d8ce","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e311c506c7592695f88d8ce: done: 0.015192 sec","job_status":"done","queue_duration_s":0.230253,"scheduling_latency_s":0.230271,"redis_calls":13,"redis_duration_s":0.0014949999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010511,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015192,"completed_at":"2026-07-01T08:09:10.051Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1373vm-test-run-nixbot-gitlab> gitlab # [ 224.663939] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fc2e9ad553084e744d81928","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fc2e9ad553084e744d81928: start","job_status":"start","queue_duration_s":0.246679,"scheduling_latency_s":0.246697}1374vm-test-run-nixbot-gitlab> gitlab # [ 224.676708] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fc2e9ad553084e744d81928","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fc2e9ad553084e744d81928: done: 0.014717 sec","job_status":"done","queue_duration_s":0.246679,"scheduling_latency_s":0.246697,"redis_calls":13,"redis_duration_s":0.001475,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014717,"completed_at":"2026-07-01T08:09:10.067Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1375vm-test-run-nixbot-gitlab> gitlab # [ 224.716247] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a69a2ad5edcf8ee7b0c6a78d","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a69a2ad5edcf8ee7b0c6a78d: start","job_status":"start","queue_duration_s":0.262418,"scheduling_latency_s":0.262435}1376vm-test-run-nixbot-gitlab> gitlab # [ 224.728757] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.087Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a69a2ad5edcf8ee7b0c6a78d","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a69a2ad5edcf8ee7b0c6a78d: done: 0.018518 sec","job_status":"done","queue_duration_s":0.262418,"scheduling_latency_s":0.262435,"redis_calls":13,"redis_duration_s":0.0016070000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000577,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010653,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018518,"completed_at":"2026-07-01T08:09:10.087Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1377vm-test-run-nixbot-gitlab> gitlab # [ 224.768496] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.088Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd7a29b0d4077e0e08ef8f9b","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd7a29b0d4077e0e08ef8f9b: start","job_status":"start","queue_duration_s":0.282,"scheduling_latency_s":0.282017}1378vm-test-run-nixbot-gitlab> gitlab # [ 224.781056] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd7a29b0d4077e0e08ef8f9b","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd7a29b0d4077e0e08ef8f9b: done: 0.014752 sec","job_status":"done","queue_duration_s":0.282,"scheduling_latency_s":0.282017,"redis_calls":13,"redis_duration_s":0.001424,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000902,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014752,"completed_at":"2026-07-01T08:09:10.103Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1379vm-test-run-nixbot-gitlab> gitlab # [ 224.821214] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.104Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d25788d0aa63cae01f9c90d","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d25788d0aa63cae01f9c90d: start","job_status":"start","queue_duration_s":0.297778,"scheduling_latency_s":0.297797}1380vm-test-run-nixbot-gitlab> gitlab # [ 224.833595] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.119Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d25788d0aa63cae01f9c90d","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d25788d0aa63cae01f9c90d: done: 0.015315 sec","job_status":"done","queue_duration_s":0.297778,"scheduling_latency_s":0.297797,"redis_calls":13,"redis_duration_s":0.001727,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00117,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015315,"completed_at":"2026-07-01T08:09:10.119Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1381vm-test-run-nixbot-gitlab> gitlab # [ 224.872935] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8217112e85e9de96f774c644","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8217112e85e9de96f774c644: start","job_status":"start","queue_duration_s":0.314322,"scheduling_latency_s":0.314339}1382vm-test-run-nixbot-gitlab> gitlab # [ 224.885498] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.136Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8217112e85e9de96f774c644","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8217112e85e9de96f774c644: done: 0.015869 sec","job_status":"done","queue_duration_s":0.314322,"scheduling_latency_s":0.314339,"redis_calls":13,"redis_duration_s":0.001497,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015869,"completed_at":"2026-07-01T08:09:10.136Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1383vm-test-run-nixbot-gitlab> gitlab # [ 224.925052] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4df80bc5b21f0294484c1a34","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4df80bc5b21f0294484c1a34: start","job_status":"start","queue_duration_s":0.33141,"scheduling_latency_s":0.331429}1384vm-test-run-nixbot-gitlab> gitlab # [ 224.937901] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4df80bc5b21f0294484c1a34","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4df80bc5b21f0294484c1a34: done: 0.014607 sec","job_status":"done","queue_duration_s":0.33141,"scheduling_latency_s":0.331429,"redis_calls":13,"redis_duration_s":0.001492,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000956,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014607,"completed_at":"2026-07-01T08:09:10.152Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1385vm-test-run-nixbot-gitlab> gitlab # [ 224.977622] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.153Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c77d4dcf88eacf2d6f743a0","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c77d4dcf88eacf2d6f743a0: start","job_status":"start","queue_duration_s":0.347279,"scheduling_latency_s":0.347296}1386vm-test-run-nixbot-gitlab> gitlab # [ 224.991626] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c77d4dcf88eacf2d6f743a0","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c77d4dcf88eacf2d6f743a0: done: 0.015067 sec","job_status":"done","queue_duration_s":0.347279,"scheduling_latency_s":0.347296,"redis_calls":13,"redis_duration_s":0.0015119999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000959,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010421,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015067,"completed_at":"2026-07-01T08:09:10.168Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1387vm-test-run-nixbot-gitlab> gitlab # [ 225.032271] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.169Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c2e2fe1b375d0f5b4fe4070","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c2e2fe1b375d0f5b4fe4070: start","job_status":"start","queue_duration_s":0.363498,"scheduling_latency_s":0.363516}1388vm-test-run-nixbot-gitlab> gitlab # [ 225.044817] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.184Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c2e2fe1b375d0f5b4fe4070","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c2e2fe1b375d0f5b4fe4070: done: 0.01446 sec","job_status":"done","queue_duration_s":0.363498,"scheduling_latency_s":0.363516,"redis_calls":13,"redis_duration_s":0.001475,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000959,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01446,"completed_at":"2026-07-01T08:09:10.184Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1389vm-test-run-nixbot-gitlab> gitlab # [ 225.084504] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b3cb2279071b663c89bfcc9","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b3cb2279071b663c89bfcc9: start","job_status":"start","queue_duration_s":0.379252,"scheduling_latency_s":0.379271}1390vm-test-run-nixbot-gitlab> gitlab # [ 225.097240] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.200Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b3cb2279071b663c89bfcc9","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b3cb2279071b663c89bfcc9: done: 0.014813 sec","job_status":"done","queue_duration_s":0.379252,"scheduling_latency_s":0.379271,"redis_calls":13,"redis_duration_s":0.00155,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010489,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014813,"completed_at":"2026-07-01T08:09:10.200Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1391vm-test-run-nixbot-gitlab> gitlab # [ 225.136722] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"475161d5ac7523a1760e9a6a","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-475161d5ac7523a1760e9a6a: start","job_status":"start","queue_duration_s":0.395137,"scheduling_latency_s":0.395155}1392vm-test-run-nixbot-gitlab> gitlab # [ 225.149038] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.216Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"475161d5ac7523a1760e9a6a","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-475161d5ac7523a1760e9a6a: done: 0.01505 sec","job_status":"done","queue_duration_s":0.395137,"scheduling_latency_s":0.395155,"redis_calls":13,"redis_duration_s":0.0014429999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000927,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01505,"completed_at":"2026-07-01T08:09:10.216Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1393vm-test-run-nixbot-gitlab> gitlab # [ 225.188846] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.217Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"170869c7d815d2f5f31485e8","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-170869c7d815d2f5f31485e8: start","job_status":"start","queue_duration_s":0.41147,"scheduling_latency_s":0.411489}1394vm-test-run-nixbot-gitlab> gitlab # [ 225.202580] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"170869c7d815d2f5f31485e8","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-170869c7d815d2f5f31485e8: done: 0.014743 sec","job_status":"done","queue_duration_s":0.41147,"scheduling_latency_s":0.411489,"redis_calls":13,"redis_duration_s":0.0014789999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014743,"completed_at":"2026-07-01T08:09:10.232Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1395vm-test-run-nixbot-gitlab> gitlab # [ 225.242556] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"387ff6f343824c793b25257c","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-387ff6f343824c793b25257c: start","job_status":"start","queue_duration_s":0.427246,"scheduling_latency_s":0.427263}1396vm-test-run-nixbot-gitlab> gitlab # [ 225.255253] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.248Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"387ff6f343824c793b25257c","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-387ff6f343824c793b25257c: done: 0.015109 sec","job_status":"done","queue_duration_s":0.427246,"scheduling_latency_s":0.427263,"redis_calls":13,"redis_duration_s":0.001535,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015109,"completed_at":"2026-07-01T08:09:10.248Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1397vm-test-run-nixbot-gitlab> gitlab # [ 225.295264] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.249Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b07372ac1de310bf0c712b06","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b07372ac1de310bf0c712b06: start","job_status":"start","queue_duration_s":0.443427,"scheduling_latency_s":0.443444}1398vm-test-run-nixbot-gitlab> gitlab # [ 225.307914] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b07372ac1de310bf0c712b06","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b07372ac1de310bf0c712b06: done: 0.014761 sec","job_status":"done","queue_duration_s":0.443427,"scheduling_latency_s":0.443444,"redis_calls":13,"redis_duration_s":0.001459,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014761,"completed_at":"2026-07-01T08:09:10.264Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1399vm-test-run-nixbot-gitlab> gitlab # [ 225.347762] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.265Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02c0e7ed806eb23573949c3a","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02c0e7ed806eb23573949c3a: start","job_status":"start","queue_duration_s":0.459362,"scheduling_latency_s":0.459379}1400vm-test-run-nixbot-gitlab> gitlab # [ 225.360602] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.280Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02c0e7ed806eb23573949c3a","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02c0e7ed806eb23573949c3a: done: 0.014846 sec","job_status":"done","queue_duration_s":0.459362,"scheduling_latency_s":0.459379,"redis_calls":13,"redis_duration_s":0.001505,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000586,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010281,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014846,"completed_at":"2026-07-01T08:09:10.280Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1401vm-test-run-nixbot-gitlab> gitlab # [ 225.400898] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.281Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b415a3da24ab651b380e29f","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b415a3da24ab651b380e29f: start","job_status":"start","queue_duration_s":0.475406,"scheduling_latency_s":0.475423}1402vm-test-run-nixbot-gitlab> gitlab # [ 225.413435] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b415a3da24ab651b380e29f","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b415a3da24ab651b380e29f: done: 0.01475 sec","job_status":"done","queue_duration_s":0.475406,"scheduling_latency_s":0.475423,"redis_calls":13,"redis_duration_s":0.0014980000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01475,"completed_at":"2026-07-01T08:09:10.296Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1403vm-test-run-nixbot-gitlab> gitlab # [ 225.453695] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.297Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10c015bd84f58fe6cc815dff","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10c015bd84f58fe6cc815dff: start","job_status":"start","queue_duration_s":0.491352,"scheduling_latency_s":0.491369}1404vm-test-run-nixbot-gitlab> gitlab # [ 225.466334] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.312Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10c015bd84f58fe6cc815dff","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10c015bd84f58fe6cc815dff: done: 0.014673 sec","job_status":"done","queue_duration_s":0.491352,"scheduling_latency_s":0.491369,"redis_calls":13,"redis_duration_s":0.0014810000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010344,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014673,"completed_at":"2026-07-01T08:09:10.312Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1405vm-test-run-nixbot-gitlab> gitlab # [ 225.506882] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.313Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2195fe7ededd62d3712edfe4","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2195fe7ededd62d3712edfe4: start","job_status":"start","queue_duration_s":0.507224,"scheduling_latency_s":0.507241}1406vm-test-run-nixbot-gitlab> gitlab # [ 225.519776] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2195fe7ededd62d3712edfe4","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2195fe7ededd62d3712edfe4: done: 0.016546 sec","job_status":"done","queue_duration_s":0.507224,"scheduling_latency_s":0.507241,"redis_calls":13,"redis_duration_s":0.001484,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010386,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016546,"completed_at":"2026-07-01T08:09:10.330Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1407vm-test-run-nixbot-gitlab> gitlab # [ 225.560743] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2114ebba9c999603285df1c","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2114ebba9c999603285df1c: start","job_status":"start","queue_duration_s":0.524863,"scheduling_latency_s":0.52488}1408vm-test-run-nixbot-gitlab> gitlab # [ 225.573348] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.345Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2114ebba9c999603285df1c","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2114ebba9c999603285df1c: done: 0.014611 sec","job_status":"done","queue_duration_s":0.524863,"scheduling_latency_s":0.52488,"redis_calls":13,"redis_duration_s":0.001454,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000911,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014611,"completed_at":"2026-07-01T08:09:10.345Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1409vm-test-run-nixbot-gitlab> gitlab # [ 225.614013] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1b2d0f49c87ff30f0308032","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1b2d0f49c87ff30f0308032: start","job_status":"start","queue_duration_s":0.540713,"scheduling_latency_s":0.54073}1410vm-test-run-nixbot-gitlab> gitlab # [ 225.626815] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.361Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1b2d0f49c87ff30f0308032","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1b2d0f49c87ff30f0308032: done: 0.014855 sec","job_status":"done","queue_duration_s":0.540713,"scheduling_latency_s":0.54073,"redis_calls":13,"redis_duration_s":0.0014889999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014855,"completed_at":"2026-07-01T08:09:10.361Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1411vm-test-run-nixbot-gitlab> gitlab # [ 225.666779] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.363Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72b9941e89c12e67edbb9845","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72b9941e89c12e67edbb9845: start","job_status":"start","queue_duration_s":0.556887,"scheduling_latency_s":0.556905}1412vm-test-run-nixbot-gitlab> gitlab # [ 225.679682] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72b9941e89c12e67edbb9845","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72b9941e89c12e67edbb9845: done: 0.014756 sec","job_status":"done","queue_duration_s":0.556887,"scheduling_latency_s":0.556905,"redis_calls":13,"redis_duration_s":0.001446,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014756,"completed_at":"2026-07-01T08:09:10.377Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1413vm-test-run-nixbot-gitlab> gitlab # [ 225.719856] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.379Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39ee876f9f9c4d2dc7276ff4","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39ee876f9f9c4d2dc7276ff4: start","job_status":"start","queue_duration_s":0.572862,"scheduling_latency_s":0.572878}1414vm-test-run-nixbot-gitlab> gitlab # [ 225.732654] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.393Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39ee876f9f9c4d2dc7276ff4","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39ee876f9f9c4d2dc7276ff4: done: 0.014623 sec","job_status":"done","queue_duration_s":0.572862,"scheduling_latency_s":0.572878,"redis_calls":13,"redis_duration_s":0.00145,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014623,"completed_at":"2026-07-01T08:09:10.393Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1415vm-test-run-nixbot-gitlab> gitlab # [ 225.773132] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8654881c52666bd735706852","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8654881c52666bd735706852: start","job_status":"start","queue_duration_s":0.588719,"scheduling_latency_s":0.588736}1416vm-test-run-nixbot-gitlab> gitlab # [ 225.785946] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8654881c52666bd735706852","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8654881c52666bd735706852: done: 0.015464 sec","job_status":"done","queue_duration_s":0.588719,"scheduling_latency_s":0.588736,"redis_calls":13,"redis_duration_s":0.001529,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015464,"completed_at":"2026-07-01T08:09:10.410Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1417vm-test-run-nixbot-gitlab> gitlab # [ 225.826927] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.411Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09f2bb9ac4de532e3c24c03f","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09f2bb9ac4de532e3c24c03f: start","job_status":"start","queue_duration_s":0.605422,"scheduling_latency_s":0.605439}1418vm-test-run-nixbot-gitlab> gitlab # [ 225.839848] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.427Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09f2bb9ac4de532e3c24c03f","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09f2bb9ac4de532e3c24c03f: done: 0.015356 sec","job_status":"done","queue_duration_s":0.605422,"scheduling_latency_s":0.605439,"redis_calls":13,"redis_duration_s":0.001487,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015356,"completed_at":"2026-07-01T08:09:10.427Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419vm-test-run-nixbot-gitlab> gitlab # [ 225.879934] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dec16c4894f759597bc97127","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dec16c4894f759597bc97127: start","job_status":"start","queue_duration_s":0.621811,"scheduling_latency_s":0.621827}1420vm-test-run-nixbot-gitlab> gitlab # [ 225.892892] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.443Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dec16c4894f759597bc97127","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dec16c4894f759597bc97127: done: 0.015076 sec","job_status":"done","queue_duration_s":0.621811,"scheduling_latency_s":0.621827,"redis_calls":13,"redis_duration_s":0.0017209999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001184,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010465,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015076,"completed_at":"2026-07-01T08:09:10.443Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421vm-test-run-nixbot-gitlab> gitlab # [ 225.932935] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.444Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48f091d715db60a313d0ffcb","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48f091d715db60a313d0ffcb: start","job_status":"start","queue_duration_s":0.638012,"scheduling_latency_s":0.638029}1422vm-test-run-nixbot-gitlab> gitlab # [ 225.945649] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.459Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48f091d715db60a313d0ffcb","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48f091d715db60a313d0ffcb: done: 0.015424 sec","job_status":"done","queue_duration_s":0.638012,"scheduling_latency_s":0.638029,"redis_calls":13,"redis_duration_s":0.001659,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015424,"completed_at":"2026-07-01T08:09:10.459Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1423vm-test-run-nixbot-gitlab> gitlab # [ 225.985859] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.460Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"207343ab0fe68e63e32548b5","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-207343ab0fe68e63e32548b5: start","job_status":"start","queue_duration_s":0.65449,"scheduling_latency_s":0.654507}1424vm-test-run-nixbot-gitlab> gitlab # [ 225.998777] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.475Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"207343ab0fe68e63e32548b5","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-207343ab0fe68e63e32548b5: done: 0.014987 sec","job_status":"done","queue_duration_s":0.65449,"scheduling_latency_s":0.654507,"redis_calls":13,"redis_duration_s":0.001656,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001129,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014987,"completed_at":"2026-07-01T08:09:10.475Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425vm-test-run-nixbot-gitlab> gitlab # [ 226.039845] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.477Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a225afcd07cafe994983c4e0","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a225afcd07cafe994983c4e0: start","job_status":"start","queue_duration_s":0.670717,"scheduling_latency_s":0.670735}1426vm-test-run-nixbot-gitlab> gitlab # [ 226.053112] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.492Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a225afcd07cafe994983c4e0","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a225afcd07cafe994983c4e0: done: 0.015093 sec","job_status":"done","queue_duration_s":0.670717,"scheduling_latency_s":0.670735,"redis_calls":13,"redis_duration_s":0.00147,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015093,"completed_at":"2026-07-01T08:09:10.492Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427vm-test-run-nixbot-gitlab> gitlab # [ 226.092812] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.493Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f34eb8b573064f40e82865de","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f34eb8b573064f40e82865de: start","job_status":"start","queue_duration_s":0.686818,"scheduling_latency_s":0.686834}1428vm-test-run-nixbot-gitlab> gitlab # [ 226.105282] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f34eb8b573064f40e82865de","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f34eb8b573064f40e82865de: done: 0.014639 sec","job_status":"done","queue_duration_s":0.686818,"scheduling_latency_s":0.686834,"redis_calls":13,"redis_duration_s":0.0016419999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000707,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014639,"completed_at":"2026-07-01T08:09:10.507Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429vm-test-run-nixbot-gitlab> gitlab # [ 226.145242] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.509Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5677a44d6cf6d100d9bea0b0","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5677a44d6cf6d100d9bea0b0: start","job_status":"start","queue_duration_s":0.702668,"scheduling_latency_s":0.702684}1430vm-test-run-nixbot-gitlab> gitlab # [ 226.157761] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5677a44d6cf6d100d9bea0b0","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5677a44d6cf6d100d9bea0b0: done: 0.015201 sec","job_status":"done","queue_duration_s":0.702668,"scheduling_latency_s":0.702684,"redis_calls":13,"redis_duration_s":0.0015249999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000589,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010417,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015201,"completed_at":"2026-07-01T08:09:10.524Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1431vm-test-run-nixbot-gitlab> gitlab # [ 226.197693] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d19f0db66ca8cf33abe5db4","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d19f0db66ca8cf33abe5db4: start","job_status":"start","queue_duration_s":0.719291,"scheduling_latency_s":0.719308}1432vm-test-run-nixbot-gitlab> gitlab # [ 226.210438] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d19f0db66ca8cf33abe5db4","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d19f0db66ca8cf33abe5db4: done: 0.186212 sec","job_status":"done","queue_duration_s":0.719291,"scheduling_latency_s":0.719308,"redis_calls":13,"redis_duration_s":0.001981,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001063,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000918,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013271,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.186212,"completed_at":"2026-07-01T08:09:10.711Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433vm-test-run-nixbot-gitlab> gitlab # [ 226.251049] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bd9a7b16abeaaeaf8d89b2c","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bd9a7b16abeaaeaf8d89b2c: start","job_status":"start","queue_duration_s":0.909637,"scheduling_latency_s":0.909682}1434vm-test-run-nixbot-gitlab> gitlab # [ 226.263931] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bd9a7b16abeaaeaf8d89b2c","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bd9a7b16abeaaeaf8d89b2c: done: 0.039286 sec","job_status":"done","queue_duration_s":0.909637,"scheduling_latency_s":0.909682,"redis_calls":13,"redis_duration_s":0.00462,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.039286,"completed_at":"2026-07-01T08:09:10.755Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435vm-test-run-nixbot-gitlab> gitlab # [ 226.304176] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"311697d468bd4ec86175bbea","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-311697d468bd4ec86175bbea: start","job_status":"start","queue_duration_s":0.951879,"scheduling_latency_s":0.951911}1436vm-test-run-nixbot-gitlab> gitlab # [ 226.316590] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"311697d468bd4ec86175bbea","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-311697d468bd4ec86175bbea: done: 0.027681 sec","job_status":"done","queue_duration_s":0.951879,"scheduling_latency_s":0.951911,"redis_calls":13,"redis_duration_s":0.003431,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001229,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002202,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027681,"completed_at":"2026-07-01T08:09:10.785Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437vm-test-run-nixbot-gitlab> gitlab # [ 226.356590] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e5550831aa31c2064f16370","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e5550831aa31c2064f16370: start","job_status":"start","queue_duration_s":0.981654,"scheduling_latency_s":0.981678}1438vm-test-run-nixbot-gitlab> gitlab # [ 226.369053] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e5550831aa31c2064f16370","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e5550831aa31c2064f16370: done: 0.032212 sec","job_status":"done","queue_duration_s":0.981654,"scheduling_latency_s":0.981678,"redis_calls":13,"redis_duration_s":0.011748999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.009519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00223,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032212,"completed_at":"2026-07-01T08:09:10.820Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439vm-test-run-nixbot-gitlab> gitlab # [ 226.408948] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d76832c4219d9fb292a81a8","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d76832c4219d9fb292a81a8: start","job_status":"start","queue_duration_s":1.01717,"scheduling_latency_s":1.017198}1440vm-test-run-nixbot-gitlab> gitlab # [ 226.422490] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d76832c4219d9fb292a81a8","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d76832c4219d9fb292a81a8: done: 0.030465 sec","job_status":"done","queue_duration_s":1.01717,"scheduling_latency_s":1.017198,"redis_calls":13,"redis_duration_s":0.006955,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00115,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005805,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030465,"completed_at":"2026-07-01T08:09:10.853Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441vm-test-run-nixbot-gitlab> gitlab # [ 226.462462] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.856Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04d4354340ba11189ce453c4","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04d4354340ba11189ce453c4: start","job_status":"start","queue_duration_s":1.04984,"scheduling_latency_s":1.049867}1442vm-test-run-nixbot-gitlab> gitlab # [ 226.474965] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.878Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04d4354340ba11189ce453c4","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04d4354340ba11189ce453c4: done: 0.022648 sec","job_status":"done","queue_duration_s":1.04984,"scheduling_latency_s":1.049867,"redis_calls":13,"redis_duration_s":0.003348,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001078,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00227,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022648,"completed_at":"2026-07-01T08:09:10.878Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443vm-test-run-nixbot-gitlab> gitlab # [ 226.515078] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.880Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4814bbdc9c0b9b2993dd9c6a","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4814bbdc9c0b9b2993dd9c6a: start","job_status":"start","queue_duration_s":1.074131,"scheduling_latency_s":1.074152}1444vm-test-run-nixbot-gitlab> gitlab # [ 226.527987] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4814bbdc9c0b9b2993dd9c6a","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4814bbdc9c0b9b2993dd9c6a: done: 0.019937 sec","job_status":"done","queue_duration_s":1.074131,"scheduling_latency_s":1.074152,"redis_calls":13,"redis_duration_s":0.0031379999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001032,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019937,"completed_at":"2026-07-01T08:09:10.900Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445vm-test-run-nixbot-gitlab> gitlab # [ 226.568075] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.902Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d83dce6e6e15fa9242cc9f81","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d83dce6e6e15fa9242cc9f81: start","job_status":"start","queue_duration_s":1.095619,"scheduling_latency_s":1.095638}1446vm-test-run-nixbot-gitlab> gitlab # [ 226.580589] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d83dce6e6e15fa9242cc9f81","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d83dce6e6e15fa9242cc9f81: done: 0.01731 sec","job_status":"done","queue_duration_s":1.095619,"scheduling_latency_s":1.095638,"redis_calls":13,"redis_duration_s":0.002862,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000951,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001911,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01731,"completed_at":"2026-07-01T08:09:10.919Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1447vm-test-run-nixbot-gitlab> gitlab # [ 226.621485] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0df9397989f5f23a04eab79e","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0df9397989f5f23a04eab79e: start","job_status":"start","queue_duration_s":1.114309,"scheduling_latency_s":1.114327}1448vm-test-run-nixbot-gitlab> gitlab # [ 226.633977] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0df9397989f5f23a04eab79e","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0df9397989f5f23a04eab79e: done: 0.015704 sec","job_status":"done","queue_duration_s":1.114309,"scheduling_latency_s":1.114327,"redis_calls":13,"redis_duration_s":0.0029100000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009704,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015704,"completed_at":"2026-07-01T08:09:10.936Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449vm-test-run-nixbot-gitlab> gitlab # [ 226.674083] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.938Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6c04d8292b56e8121648d2a","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6c04d8292b56e8121648d2a: start","job_status":"start","queue_duration_s":1.13208,"scheduling_latency_s":1.132098}1450vm-test-run-nixbot-gitlab> gitlab # [ 226.686836] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.954Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6c04d8292b56e8121648d2a","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6c04d8292b56e8121648d2a: done: 0.015897 sec","job_status":"done","queue_duration_s":1.13208,"scheduling_latency_s":1.132098,"redis_calls":13,"redis_duration_s":0.002796,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000957,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001839,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015897,"completed_at":"2026-07-01T08:09:10.954Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451vm-test-run-nixbot-gitlab> gitlab # [ 226.727051] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"648b561de8e4c725eb52a29d","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-648b561de8e4c725eb52a29d: start","job_status":"start","queue_duration_s":1.149631,"scheduling_latency_s":1.149647}1452vm-test-run-nixbot-gitlab> gitlab # [ 226.739733] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.971Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"648b561de8e4c725eb52a29d","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-648b561de8e4c725eb52a29d: done: 0.015573 sec","job_status":"done","queue_duration_s":1.149631,"scheduling_latency_s":1.149647,"redis_calls":13,"redis_duration_s":0.002704,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001759,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015573,"completed_at":"2026-07-01T08:09:10.971Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453vm-test-run-nixbot-gitlab> gitlab # [ 226.779627] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cdee5eac939fc3849e4955c","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cdee5eac939fc3849e4955c: start","job_status":"start","queue_duration_s":1.166692,"scheduling_latency_s":1.166708}1454vm-test-run-nixbot-gitlab> gitlab # [ 226.792441] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cdee5eac939fc3849e4955c","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cdee5eac939fc3849e4955c: done: 0.015936 sec","job_status":"done","queue_duration_s":1.166692,"scheduling_latency_s":1.166708,"redis_calls":13,"redis_duration_s":0.002866,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000958,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001908,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015936,"completed_at":"2026-07-01T08:09:10.988Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455vm-test-run-nixbot-gitlab> gitlab # [ 226.832944] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:10.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb38e4d2dbf54ab3685b0cf4","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb38e4d2dbf54ab3685b0cf4: start","job_status":"start","queue_duration_s":1.184263,"scheduling_latency_s":1.18428}1456vm-test-run-nixbot-gitlab> gitlab # [ 226.845332] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.006Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb38e4d2dbf54ab3685b0cf4","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb38e4d2dbf54ab3685b0cf4: done: 0.015517 sec","job_status":"done","queue_duration_s":1.184263,"scheduling_latency_s":1.18428,"redis_calls":13,"redis_duration_s":0.00267,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001737,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015517,"completed_at":"2026-07-01T08:09:11.006Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457vm-test-run-nixbot-gitlab> gitlab # [ 226.884626] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d756f35b0c611a0dda0196f","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d756f35b0c611a0dda0196f: start","job_status":"start","queue_duration_s":1.20146,"scheduling_latency_s":1.201477}1458vm-test-run-nixbot-gitlab> gitlab # [ 226.897075] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.023Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d756f35b0c611a0dda0196f","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d756f35b0c611a0dda0196f: done: 0.015729 sec","job_status":"done","queue_duration_s":1.20146,"scheduling_latency_s":1.201477,"redis_calls":13,"redis_duration_s":0.002674,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001762,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015729,"completed_at":"2026-07-01T08:09:11.023Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459vm-test-run-nixbot-gitlab> gitlab # [ 226.936354] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1106dca9201d6a724b82bcc","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1106dca9201d6a724b82bcc: start","job_status":"start","queue_duration_s":1.219081,"scheduling_latency_s":1.219099}1460vm-test-run-nixbot-gitlab> gitlab # [ 226.948811] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.041Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1106dca9201d6a724b82bcc","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1106dca9201d6a724b82bcc: done: 0.015851 sec","job_status":"done","queue_duration_s":1.219081,"scheduling_latency_s":1.219099,"redis_calls":13,"redis_duration_s":0.002996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002061,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015851,"completed_at":"2026-07-01T08:09:11.041Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461vm-test-run-nixbot-gitlab> gitlab # [ 226.988268] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6112e78d0767fe9a4a217e2","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6112e78d0767fe9a4a217e2: start","job_status":"start","queue_duration_s":1.236271,"scheduling_latency_s":1.236288}1462vm-test-run-nixbot-gitlab> gitlab # [ 227.000476] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.058Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6112e78d0767fe9a4a217e2","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6112e78d0767fe9a4a217e2: done: 0.015984 sec","job_status":"done","queue_duration_s":1.236271,"scheduling_latency_s":1.236288,"redis_calls":13,"redis_duration_s":0.003077,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001124,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015984,"completed_at":"2026-07-01T08:09:11.058Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463vm-test-run-nixbot-gitlab> gitlab # [ 227.040788] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.059Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"797a2138491cc77a360888b3","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-797a2138491cc77a360888b3: start","job_status":"start","queue_duration_s":1.253569,"scheduling_latency_s":1.253585}1464vm-test-run-nixbot-gitlab> gitlab # [ 227.053406] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.076Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"797a2138491cc77a360888b3","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-797a2138491cc77a360888b3: done: 0.016133 sec","job_status":"done","queue_duration_s":1.253569,"scheduling_latency_s":1.253585,"redis_calls":13,"redis_duration_s":0.00297,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00099,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009807,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016133,"completed_at":"2026-07-01T08:09:11.076Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465vm-test-run-nixbot-gitlab> gitlab # [ 227.092620] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.077Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9b5e1912c8734f634519a67","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9b5e1912c8734f634519a67: start","job_status":"start","queue_duration_s":1.271028,"scheduling_latency_s":1.271045}1466vm-test-run-nixbot-gitlab> gitlab # [ 227.105246] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.093Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9b5e1912c8734f634519a67","created_at":"2026-07-01T08:09:09.687Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-868b5fa3e32648de","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8595fed8428d8a23c23a4a81671e9c87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.806Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9b5e1912c8734f634519a67: done: 0.015576 sec","job_status":"done","queue_duration_s":1.271028,"scheduling_latency_s":1.271045,"redis_calls":13,"redis_duration_s":0.0026579999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001733,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009637,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015576,"completed_at":"2026-07-01T08:09:11.092Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1467vm-test-run-nixbot-gitlab> gitlab # [ 227.144613] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.094Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dae3a0ff1b551e026062a5b0","created_at":"2026-07-01T08:09:09.956Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-444d3087f5254089","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bf4fe4e532e1b7b69c7983ac0e48dcda","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.969Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dae3a0ff1b551e026062a5b0: start","job_status":"start","queue_duration_s":1.125164,"scheduling_latency_s":1.125182}1468vm-test-run-nixbot-gitlab> gitlab # [ 227.157042] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.112Z","message":"No existing merge request to be cleaned up."}1469vm-test-run-nixbot-gitlab> gitlab # [ 227.158717] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.127Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dae3a0ff1b551e026062a5b0","created_at":"2026-07-01T08:09:09.956Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-444d3087f5254089","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bf4fe4e532e1b7b69c7983ac0e48dcda","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.969Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dae3a0ff1b551e026062a5b0: done: 0.033425 sec","job_status":"done","queue_duration_s":1.125164,"scheduling_latency_s":1.125182,"redis_calls":13,"redis_duration_s":0.003324,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000585,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001177,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001562,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019201,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034568,"duration_s":0.033425,"completed_at":"2026-07-01T08:09:11.127Z","load_balancing_strategy":"primary","db_duration_s":0.006645,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1470vm-test-run-nixbot-gitlab> gitlab # [ 227.199045] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.129Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bb81fc2959ccaae73388caeb","created_at":"2026-07-01T08:09:09.956Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-444d3087f5254089","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bf4fe4e532e1b7b69c7983ac0e48dcda","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.969Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bb81fc2959ccaae73388caeb: start","job_status":"start","queue_duration_s":1.160095,"scheduling_latency_s":1.160111}1471vm-test-run-nixbot-gitlab> gitlab # [ 227.211640] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.140Z","message":"No existing merge request to be cleaned up."}1472vm-test-run-nixbot-gitlab> gitlab # [ 227.213416] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.148Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bb81fc2959ccaae73388caeb","created_at":"2026-07-01T08:09:09.956Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-444d3087f5254089","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bf4fe4e532e1b7b69c7983ac0e48dcda","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.969Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bb81fc2959ccaae73388caeb: done: 0.019616 sec","job_status":"done","queue_duration_s":1.160095,"scheduling_latency_s":1.160111,"redis_calls":10,"redis_duration_s":0.002864,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00178,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019616,"completed_at":"2026-07-01T08:09:11.148Z","load_balancing_strategy":"primary","db_duration_s":0.002748,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473vm-test-run-nixbot-gitlab> gitlab # [ 227.252865] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.150Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c66cb914b2e9cca55e169d14","created_at":"2026-07-01T08:09:09.956Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-444d3087f5254089","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bf4fe4e532e1b7b69c7983ac0e48dcda","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.969Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c66cb914b2e9cca55e169d14: start","job_status":"start","queue_duration_s":1.181077,"scheduling_latency_s":1.181094}1474vm-test-run-nixbot-gitlab> gitlab # [ 227.265290] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.161Z","message":"No existing merge request to be cleaned up."}1475vm-test-run-nixbot-gitlab> gitlab # [ 227.267012] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.170Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c66cb914b2e9cca55e169d14","created_at":"2026-07-01T08:09:09.956Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-444d3087f5254089","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bf4fe4e532e1b7b69c7983ac0e48dcda","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.969Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c66cb914b2e9cca55e169d14: done: 0.01978 sec","job_status":"done","queue_duration_s":1.181077,"scheduling_latency_s":1.181094,"redis_calls":10,"redis_duration_s":0.002805,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001065,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00174,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012411,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01978,"completed_at":"2026-07-01T08:09:11.170Z","load_balancing_strategy":"primary","db_duration_s":0.002571,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1476vm-test-run-nixbot-gitlab> gitlab # [ 227.305704] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.171Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b6e79e942f9a8451455e5c33","created_at":"2026-07-01T08:09:09.956Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-444d3087f5254089","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bf4fe4e532e1b7b69c7983ac0e48dcda","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.969Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b6e79e942f9a8451455e5c33: start","job_status":"start","queue_duration_s":1.202254,"scheduling_latency_s":1.20227}1477vm-test-run-nixbot-gitlab> gitlab # [ 227.317945] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.182Z","message":"No existing merge request to be cleaned up."}1478vm-test-run-nixbot-gitlab> gitlab # [ 227.319563] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:09:11.191Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b6e79e942f9a8451455e5c33","created_at":"2026-07-01T08:09:09.956Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-444d3087f5254089","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bf4fe4e532e1b7b69c7983ac0e48dcda","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:09:09.969Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b6e79e942f9a8451455e5c33: done: 0.019616 sec","job_status":"done","queue_duration_s":1.202254,"scheduling_latency_s":1.20227,"redis_calls":10,"redis_duration_s":0.002444,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001022,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001422,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019616,"completed_at":"2026-07-01T08:09:11.191Z","load_balancing_strategy":"primary","db_duration_s":0.002479,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479vm-test-run-nixbot-gitlab> gitlab # [ 255.725105] setup-gitlab-start[3304]: W, [2026-07-01T08:09:41.594085 #3304] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1480vm-test-run-nixbot-gitlab> gitlab # [ 278.059971] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:03.928Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"2bf335f8c1a3ff48cf1d0248","created_at":"2026-07-01T08:10:03.914Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f03aa8d1b94b62805e58646af6e8f614","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-01T08:10:03.924Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-2bf335f8c1a3ff48cf1d0248: start","job_status":"start","queue_duration_s":0.004004,"scheduling_latency_s":0.004065}1481vm-test-run-nixbot-gitlab> gitlab # [ 278.321219] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:04.190Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"2bf335f8c1a3ff48cf1d0248","created_at":"2026-07-01T08:10:03.914Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f03aa8d1b94b62805e58646af6e8f614","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-01T08:10:03.924Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-2bf335f8c1a3ff48cf1d0248: done: 0.262181 sec","job_status":"done","queue_duration_s":0.004004,"scheduling_latency_s":0.004065,"redis_calls":14,"redis_duration_s":0.031922,"redis_read_bytes":901,"redis_write_bytes":1211,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.019491,"redis_feature_flag_read_bytes":896,"redis_feature_flag_write_bytes":470,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006984,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005447,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055327,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.262181,"completed_at":"2026-07-01T08:10:04.189Z","load_balancing_strategy":"primary","db_duration_s":0.017375,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1482vm-test-run-nixbot-gitlab> gitlab # [ 278.415221] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:04.199Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"2fe27f8cab264109b9d3ab9e","created_at":"2026-07-01T08:10:03.947Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ac7cae62e3d313aba9c973fbbe8bf0d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:03.962Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-2fe27f8cab264109b9d3ab9e: start","job_status":"start","queue_duration_s":0.236825,"scheduling_latency_s":0.236874}1483vm-test-run-nixbot-gitlab> gitlab # [ 278.668123] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:04.536Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"2fe27f8cab264109b9d3ab9e","created_at":"2026-07-01T08:10:03.947Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ac7cae62e3d313aba9c973fbbe8bf0d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:03.962Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-2fe27f8cab264109b9d3ab9e: done: 0.33759 sec","job_status":"done","queue_duration_s":0.236825,"scheduling_latency_s":0.236874,"redis_calls":9,"redis_duration_s":0.03964,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003742,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.035898,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.07,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.07,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.063839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0012395,"exclusive_lock_hold_duration_s":0.24807182,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.33759,"completed_at":"2026-07-01T08:10:04.536Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.274845,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1484vm-test-run-nixbot-gitlab> gitlab # [ 278.764417] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:04.551Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"190fd115b30c0c1d105757cc","created_at":"2026-07-01T08:10:03.991Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc6c479468a65a82fa706873dd2635ad","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.003Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-190fd115b30c0c1d105757cc: start","job_status":"start","queue_duration_s":0.547289,"scheduling_latency_s":0.547338}1485vm-test-run-nixbot-gitlab> gitlab # [ 278.782318] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:04.571Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1486vm-test-run-nixbot-gitlab> gitlab # [ 278.786197] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:04.590Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"190fd115b30c0c1d105757cc","created_at":"2026-07-01T08:10:03.991Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc6c479468a65a82fa706873dd2635ad","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.003Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-190fd115b30c0c1d105757cc: done: 0.039548 sec","job_status":"done","queue_duration_s":0.547289,"scheduling_latency_s":0.547338,"redis_calls":7,"redis_duration_s":0.004304,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003525,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000779,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024496,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039548,"completed_at":"2026-07-01T08:10:04.590Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487vm-test-run-nixbot-gitlab> gitlab # [ 278.828840] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:04.593Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"e66fb1b823e8f3ec7d854120","created_at":"2026-07-01T08:10:04.039Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"49d7c094c9aba310c2df010bc918dc2e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.047Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-e66fb1b823e8f3ec7d854120: start","job_status":"start","queue_duration_s":0.546237,"scheduling_latency_s":0.546279}1488vm-test-run-nixbot-gitlab> gitlab # [ 278.842401] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:04.635Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"e66fb1b823e8f3ec7d854120","created_at":"2026-07-01T08:10:04.039Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"49d7c094c9aba310c2df010bc918dc2e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.047Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-e66fb1b823e8f3ec7d854120: done: 0.041842 sec","job_status":"done","queue_duration_s":0.546237,"scheduling_latency_s":0.546279,"redis_calls":9,"redis_duration_s":0.00276,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00145,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00131,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00107958,"exclusive_lock_hold_duration_s":0.00552032,"duration_s":0.041842,"completed_at":"2026-07-01T08:10:04.635Z","load_balancing_strategy":"primary","db_duration_s":0.002199,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489vm-test-run-nixbot-gitlab> gitlab # [ 278.884061] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:04.637Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"0dbb9ed4bc3184a169b6d55d","created_at":"2026-07-01T08:10:04.168Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c803dfd9d1444e5d669e4f31ca14fd4","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.176Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-0dbb9ed4bc3184a169b6d55d: start","job_status":"start","queue_duration_s":0.460872,"scheduling_latency_s":0.460903}1490vm-test-run-nixbot-gitlab> gitlab # [ 278.919379] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:04.788Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"0dbb9ed4bc3184a169b6d55d","created_at":"2026-07-01T08:10:04.168Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c803dfd9d1444e5d669e4f31ca14fd4","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.176Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-0dbb9ed4bc3184a169b6d55d: done: 0.151103 sec","job_status":"done","queue_duration_s":0.460872,"scheduling_latency_s":0.460903,"redis_calls":7,"redis_duration_s":0.005778999999999999,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005122,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000657,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.048,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.151103,"completed_at":"2026-07-01T08:10:04.788Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.050175,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491vm-test-run-nixbot-gitlab> gitlab # [ 278.967994] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:04.795Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"592189e4a8775e23d8be7a56","created_at":"2026-07-01T08:10:04.187Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53de609042080072e1493d47ebdb86d9","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.239Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-592189e4a8775e23d8be7a56: start","job_status":"start","queue_duration_s":0.555718,"scheduling_latency_s":0.555769}1492vm-test-run-nixbot-gitlab> gitlab # [ 279.026996] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:04.896Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"592189e4a8775e23d8be7a56","created_at":"2026-07-01T08:10:04.187Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53de609042080072e1493d47ebdb86d9","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.239Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-592189e4a8775e23d8be7a56: done: 0.100538 sec","job_status":"done","queue_duration_s":0.555718,"scheduling_latency_s":0.555769,"redis_calls":10,"redis_duration_s":0.014078,"redis_read_bytes":9,"redis_write_bytes":1087,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000809,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011043,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002226,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00102808,"duration_s":0.100538,"completed_at":"2026-07-01T08:10:04.896Z","load_balancing_strategy":"primary","db_duration_s":0.003115,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493vm-test-run-nixbot-gitlab> gitlab # [ 279.093845] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:04.902Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"241b7ee86eabd1cf5f56959b","created_at":"2026-07-01T08:10:04.247Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4880789cd816263a8b2249586d362f8a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.253Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-241b7ee86eabd1cf5f56959b: start","job_status":"start","queue_duration_s":0.648746,"scheduling_latency_s":0.648795}1494vm-test-run-nixbot-gitlab> gitlab # [ 279.664625] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:05.534Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"241b7ee86eabd1cf5f56959b","created_at":"2026-07-01T08:10:04.247Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4880789cd816263a8b2249586d362f8a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.253Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-241b7ee86eabd1cf5f56959b: done: 0.631692 sec","job_status":"done","queue_duration_s":0.648746,"scheduling_latency_s":0.648795,"redis_calls":11,"redis_duration_s":0.0211,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.006424,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008414,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006262,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":124,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":124,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.118,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":124,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.118,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.381935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00523878,"exclusive_lock_hold_duration_s":0.55052364,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.631692,"completed_at":"2026-07-01T08:10:05.534Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.235086,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495vm-test-run-nixbot-gitlab> gitlab # [ 279.758210] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:05.536Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"bc438b76767a4cee3328ba91","created_at":"2026-07-01T08:10:04.261Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8db04a1006449a278f9dc87843a5533a","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.271Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-bc438b76767a4cee3328ba91: start","job_status":"start","queue_duration_s":1.264242,"scheduling_latency_s":1.264265}1496vm-test-run-nixbot-gitlab> gitlab # [ 279.773790] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:05.557Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"bc438b76767a4cee3328ba91","created_at":"2026-07-01T08:10:04.261Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8db04a1006449a278f9dc87843a5533a","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.271Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-bc438b76767a4cee3328ba91: done: 0.02193 sec","job_status":"done","queue_duration_s":1.264242,"scheduling_latency_s":1.264265,"redis_calls":12,"redis_duration_s":0.003029,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000569,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000541,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001361,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000558,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034196,"duration_s":0.02193,"completed_at":"2026-07-01T08:10:05.557Z","load_balancing_strategy":"primary","db_duration_s":0.000535,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497vm-test-run-nixbot-gitlab> gitlab # [ 279.817367] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:05.559Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"2bd5d3719ddd914d96204bc4","created_at":"2026-07-01T08:10:04.317Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31e05ab36bffb97327d9f8dc0ff97c19","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.322Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-2bd5d3719ddd914d96204bc4: start","job_status":"start","queue_duration_s":1.236477,"scheduling_latency_s":1.236493}1498vm-test-run-nixbot-gitlab> gitlab # [ 280.343023] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:06.210Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"2bd5d3719ddd914d96204bc4","created_at":"2026-07-01T08:10:04.317Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31e05ab36bffb97327d9f8dc0ff97c19","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.322Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-2bd5d3719ddd914d96204bc4: done: 0.651592 sec","job_status":"done","queue_duration_s":1.236477,"scheduling_latency_s":1.236493,"redis_calls":571,"redis_duration_s":0.315627,"redis_read_bytes":569,"redis_write_bytes":49487,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005281,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.310346,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.378604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.651592,"completed_at":"2026-07-01T08:10:06.210Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1499vm-test-run-nixbot-gitlab> gitlab # [ 280.433657] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:06.217Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"ce90cb1ce5e6d71fa8fd22d2","created_at":"2026-07-01T08:10:04.335Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebfd5431e64052292c27d55c4ff7e6a9","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.349Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-ce90cb1ce5e6d71fa8fd22d2: start","job_status":"start","queue_duration_s":1.868028,"scheduling_latency_s":1.86808}1500vm-test-run-nixbot-gitlab> gitlab # [ 280.451061] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:06.306Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"ce90cb1ce5e6d71fa8fd22d2","created_at":"2026-07-01T08:10:04.335Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebfd5431e64052292c27d55c4ff7e6a9","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.349Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-ce90cb1ce5e6d71fa8fd22d2: done: 0.0892 sec","job_status":"done","queue_duration_s":1.868028,"scheduling_latency_s":1.86808,"redis_calls":10,"redis_duration_s":0.015297,"redis_read_bytes":9,"redis_write_bytes":1179,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001874,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009784,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003639,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":238,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040552,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00200716,"duration_s":0.0892,"completed_at":"2026-07-01T08:10:06.306Z","load_balancing_strategy":"primary","db_duration_s":0.009218,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1501vm-test-run-nixbot-gitlab> gitlab # [ 280.495103] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:06.309Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"016b37535c4a4e8ece979db6","created_at":"2026-07-01T08:10:04.360Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a988c8b5a803e9ed3f84ee3d9dad8418","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.368Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-016b37535c4a4e8ece979db6: start","job_status":"start","queue_duration_s":1.940365,"scheduling_latency_s":1.940405}1502vm-test-run-nixbot-gitlab> gitlab # [ 280.553497] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:06.422Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"016b37535c4a4e8ece979db6","created_at":"2026-07-01T08:10:04.360Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a988c8b5a803e9ed3f84ee3d9dad8418","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.368Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-016b37535c4a4e8ece979db6: done: 0.112856 sec","job_status":"done","queue_duration_s":1.940365,"scheduling_latency_s":1.940405,"redis_calls":7,"redis_duration_s":0.004946,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003106,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00184,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.063,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.063,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0253,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.112856,"completed_at":"2026-07-01T08:10:06.422Z","load_balancing_strategy":"primary","db_duration_s":0.063416,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1503vm-test-run-nixbot-gitlab> gitlab # [ 280.629107] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:06.434Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b169bfd3fb533b85a2bcba5a","created_at":"2026-07-01T08:10:04.379Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4f37df37c5daea9ba20c66ff0a723ae","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.393Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b169bfd3fb533b85a2bcba5a: start","job_status":"start","queue_duration_s":2.041621,"scheduling_latency_s":2.041667}1504vm-test-run-nixbot-gitlab> gitlab # [ 280.650953] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:06.519Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b169bfd3fb533b85a2bcba5a","created_at":"2026-07-01T08:10:04.379Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4f37df37c5daea9ba20c66ff0a723ae","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.393Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b169bfd3fb533b85a2bcba5a: done: 0.08528 sec","job_status":"done","queue_duration_s":2.041621,"scheduling_latency_s":2.041667,"redis_calls":9,"redis_duration_s":0.024986,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.015107,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.009879,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029695,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00244468,"exclusive_lock_hold_duration_s":0.00950844,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.08528,"completed_at":"2026-07-01T08:10:06.519Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006132,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505vm-test-run-nixbot-gitlab> gitlab # [ 280.705735] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:06.527Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"ec441208ef6506241f5e7fa0","created_at":"2026-07-01T08:10:04.404Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c16f9e292e230082de4afc8e23fb5197","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.413Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-ec441208ef6506241f5e7fa0: start","job_status":"start","queue_duration_s":2.114563,"scheduling_latency_s":2.114659}1506vm-test-run-nixbot-gitlab> gitlab # [ 280.721920] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:06.575Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1507vm-test-run-nixbot-gitlab> gitlab # [ 280.760864] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:06.629Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"ec441208ef6506241f5e7fa0","created_at":"2026-07-01T08:10:04.404Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c16f9e292e230082de4afc8e23fb5197","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.413Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-ec441208ef6506241f5e7fa0: done: 0.101488 sec","job_status":"done","queue_duration_s":2.114563,"scheduling_latency_s":2.114659,"redis_calls":7,"redis_duration_s":0.005568,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003967,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001601,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.101488,"completed_at":"2026-07-01T08:10:06.628Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1508vm-test-run-nixbot-gitlab> gitlab # [ 280.811011] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:06.640Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"6b13c5747d5a24c610f7a6b2","created_at":"2026-07-01T08:10:04.421Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e1f4dbb8ece63ad4ec5d32cfe734d3f","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.429Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-6b13c5747d5a24c610f7a6b2: start","job_status":"start","queue_duration_s":2.211393,"scheduling_latency_s":2.211431}1509vm-test-run-nixbot-gitlab> gitlab # [ 280.889675] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:06.758Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"6b13c5747d5a24c610f7a6b2","created_at":"2026-07-01T08:10:04.421Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e1f4dbb8ece63ad4ec5d32cfe734d3f","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.429Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-6b13c5747d5a24c610f7a6b2: done: 0.117896 sec","job_status":"done","queue_duration_s":2.211393,"scheduling_latency_s":2.211431,"redis_calls":10,"redis_duration_s":0.01816,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012563,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005597,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01754704,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.117896,"completed_at":"2026-07-01T08:10:06.758Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014388,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1510vm-test-run-nixbot-gitlab> gitlab # [ 280.984469] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:06.762Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"eed168b491b0ef02df90180d","created_at":"2026-07-01T08:10:04.445Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4681412bf6a514555b1d07e3012327f0","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.466Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-eed168b491b0ef02df90180d: start","job_status":"start","queue_duration_s":2.296551,"scheduling_latency_s":2.296597}1511vm-test-run-nixbot-gitlab> gitlab # [ 281.074038] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:06.943Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"eed168b491b0ef02df90180d","created_at":"2026-07-01T08:10:04.445Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4681412bf6a514555b1d07e3012327f0","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.466Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-eed168b491b0ef02df90180d: done: 0.180652 sec","job_status":"done","queue_duration_s":2.296551,"scheduling_latency_s":2.296597,"redis_calls":10,"redis_duration_s":0.011429,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001496,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006433,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0035,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.053,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.053,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00335932,"extra.pages_deactivate_expired_deployments_cron_worker.deactivate_expired_pages_deployments":{"deactivated_deployments":0,"duration":0.00792126},"duration_s":0.180652,"completed_at":"2026-07-01T08:10:06.943Z","load_balancing_strategy":"primary","db_duration_s":0.072829,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1512vm-test-run-nixbot-gitlab> gitlab # [ 281.175580] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:06.955Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"cc8446cdc90a6312ca2b5182","created_at":"2026-07-01T08:10:04.507Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"450b953efa9114c9d85af26d4be2733e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.515Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-cc8446cdc90a6312ca2b5182: start","job_status":"start","queue_duration_s":2.439813,"scheduling_latency_s":2.439864}1513vm-test-run-nixbot-gitlab> gitlab # [ 281.194713] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:07.038Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"cc8446cdc90a6312ca2b5182","created_at":"2026-07-01T08:10:04.507Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"450b953efa9114c9d85af26d4be2733e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.515Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-cc8446cdc90a6312ca2b5182: done: 0.083037 sec","job_status":"done","queue_duration_s":2.439813,"scheduling_latency_s":2.439864,"redis_calls":7,"redis_duration_s":0.023079,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.016561,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.006518,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02547,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.083037,"completed_at":"2026-07-01T08:10:07.038Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1514vm-test-run-nixbot-gitlab> gitlab # [ 281.238049] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:07.050Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"8549751196f554e80c4d05ba","created_at":"2026-07-01T08:10:04.531Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93f1410c937bd1861384f1272ba469c8","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.644Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-8549751196f554e80c4d05ba: start","job_status":"start","queue_duration_s":2.406194,"scheduling_latency_s":2.406239}1515vm-test-run-nixbot-gitlab> gitlab # [ 281.390381] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:07.258Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"8549751196f554e80c4d05ba","created_at":"2026-07-01T08:10:04.531Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93f1410c937bd1861384f1272ba469c8","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.644Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-8549751196f554e80c4d05ba: done: 0.208016 sec","job_status":"done","queue_duration_s":2.406194,"scheduling_latency_s":2.406239,"redis_calls":20,"redis_duration_s":0.042215,"redis_read_bytes":23,"redis_write_bytes":4316,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.012781,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1065,"redis_queues_calls":4,"redis_queues_duration_s":0.00621,"redis_queues_read_bytes":6,"redis_queues_write_bytes":2021,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.01878,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":1004,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004444,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0032997,"duration_s":0.208016,"completed_at":"2026-07-01T08:10:07.258Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007901,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1516vm-test-run-nixbot-gitlab> gitlab # [ 281.481187] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:07.264Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"720d4494053ebd36357748c1","created_at":"2026-07-01T08:10:04.696Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e76f3dc1a8ced5d9032b49479a147844","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.713Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-720d4494053ebd36357748c1: start","job_status":"start","queue_duration_s":2.551054,"scheduling_latency_s":2.551105}1517vm-test-run-nixbot-gitlab> gitlab # [ 281.498526] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:07.299Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1518vm-test-run-nixbot-gitlab> gitlab # [ 281.502556] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:07.326Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"720d4494053ebd36357748c1","created_at":"2026-07-01T08:10:04.696Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e76f3dc1a8ced5d9032b49479a147844","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.713Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-720d4494053ebd36357748c1: done: 0.061942 sec","job_status":"done","queue_duration_s":2.551054,"scheduling_latency_s":2.551105,"redis_calls":7,"redis_duration_s":0.002805,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002042,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000763,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.061942,"completed_at":"2026-07-01T08:10:07.326Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1519vm-test-run-nixbot-gitlab> gitlab # [ 281.545663] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:07.333Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"44a20fee71348cfed756b768","created_at":"2026-07-01T08:10:04.747Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9b41f5c7f7ac5193c364e3f85c02e0d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.759Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-44a20fee71348cfed756b768: start","job_status":"start","queue_duration_s":2.573406,"scheduling_latency_s":2.57346}1520vm-test-run-nixbot-gitlab> gitlab # [ 281.560803] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:07.363Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1521vm-test-run-nixbot-gitlab> gitlab # [ 281.564670] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:07.398Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"44a20fee71348cfed756b768","created_at":"2026-07-01T08:10:04.747Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9b41f5c7f7ac5193c364e3f85c02e0d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.759Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-44a20fee71348cfed756b768: done: 0.065924 sec","job_status":"done","queue_duration_s":2.573406,"scheduling_latency_s":2.57346,"redis_calls":7,"redis_duration_s":0.004293,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00343,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000863,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.065924,"completed_at":"2026-07-01T08:10:07.398Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1522vm-test-run-nixbot-gitlab> gitlab # [ 281.607085] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:07.405Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"aaa867de102058029e9eade0","created_at":"2026-07-01T08:10:04.781Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4376cd6b92640a0015d4807b535f3a0f","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.798Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-aaa867de102058029e9eade0: start","job_status":"start","queue_duration_s":2.606627,"scheduling_latency_s":2.606721}1523vm-test-run-nixbot-gitlab> gitlab # [ 281.692608] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:07.561Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"aaa867de102058029e9eade0","created_at":"2026-07-01T08:10:04.781Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4376cd6b92640a0015d4807b535f3a0f","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.798Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-aaa867de102058029e9eade0: done: 0.156524 sec","job_status":"done","queue_duration_s":2.606627,"scheduling_latency_s":2.606721,"redis_calls":10,"redis_duration_s":0.021288,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003928,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012386,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004974,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0038342,"duration_s":0.156524,"completed_at":"2026-07-01T08:10:07.561Z","load_balancing_strategy":"primary","db_duration_s":0.04711,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1524vm-test-run-nixbot-gitlab> gitlab # [ 281.792136] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:07.568Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"38e956a055690517705b11f7","created_at":"2026-07-01T08:10:04.808Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de0e87d7de46c4610255f98236cd4a71","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.820Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-38e956a055690517705b11f7: start","job_status":"start","queue_duration_s":2.747443,"scheduling_latency_s":2.747497}1525vm-test-run-nixbot-gitlab> gitlab # [ 281.938561] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:07.807Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"38e956a055690517705b11f7","created_at":"2026-07-01T08:10:04.808Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de0e87d7de46c4610255f98236cd4a71","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.820Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-38e956a055690517705b11f7: done: 0.239681 sec","job_status":"done","queue_duration_s":2.747443,"scheduling_latency_s":2.747497,"redis_calls":11,"redis_duration_s":0.030434,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00572,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.015665,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.004262,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004787,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00168978,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.239681,"completed_at":"2026-07-01T08:10:07.807Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001182,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1526vm-test-run-nixbot-gitlab> gitlab # [ 282.029634] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:07.820Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"7bc2fd6ea549807155620615","created_at":"2026-07-01T08:10:04.872Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.891Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-7bc2fd6ea549807155620615: start","job_status":"start","queue_duration_s":2.929507,"scheduling_latency_s":2.929563}1527vm-test-run-nixbot-gitlab> gitlab # [ 282.455545] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:08.323Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"7bc2fd6ea549807155620615","created_at":"2026-07-01T08:10:04.872Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.891Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-7bc2fd6ea549807155620615: done: 0.502901 sec","job_status":"done","queue_duration_s":2.929507,"scheduling_latency_s":2.929563,"redis_calls":60,"redis_duration_s":0.138346,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.003678,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.126818,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00785,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.233739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.502901,"completed_at":"2026-07-01T08:10:08.323Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1528vm-test-run-nixbot-gitlab> gitlab # [ 282.550165] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:08.328Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"25f3773de8bcb319da9005d1","created_at":"2026-07-01T08:10:04.915Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea24efe368128abab4eb51c43114297a","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.921Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-25f3773de8bcb319da9005d1: start","job_status":"start","queue_duration_s":3.406282,"scheduling_latency_s":3.40634}1529vm-test-run-nixbot-gitlab> gitlab # [ 282.571585] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:08.401Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"25f3773de8bcb319da9005d1","created_at":"2026-07-01T08:10:04.915Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea24efe368128abab4eb51c43114297a","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.921Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-25f3773de8bcb319da9005d1: done: 0.073232 sec","job_status":"done","queue_duration_s":3.406282,"scheduling_latency_s":3.40634,"redis_calls":8,"redis_duration_s":0.008927000000000001,"redis_read_bytes":133,"redis_write_bytes":791,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000368,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":110,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004856,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003703,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.073232,"completed_at":"2026-07-01T08:10:08.400Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00623,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1530vm-test-run-nixbot-gitlab> gitlab # [ 282.624856] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:08.412Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"03cf716714b3ca73f20636d1","created_at":"2026-07-01T08:10:04.947Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9975f8cc1161db8a56eec40bc1b1131f","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.975Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-03cf716714b3ca73f20636d1: start","job_status":"start","queue_duration_s":3.437054,"scheduling_latency_s":3.437104}1531vm-test-run-nixbot-gitlab> gitlab # [ 282.695327] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:08.564Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"03cf716714b3ca73f20636d1","created_at":"2026-07-01T08:10:04.947Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9975f8cc1161db8a56eec40bc1b1131f","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.975Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-03cf716714b3ca73f20636d1: done: 0.152602 sec","job_status":"done","queue_duration_s":3.437054,"scheduling_latency_s":3.437104,"redis_calls":10,"redis_duration_s":0.023572000000000003,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002933,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.016188,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004451,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033816,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00322836,"duration_s":0.152602,"completed_at":"2026-07-01T08:10:08.564Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015125,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1532vm-test-run-nixbot-gitlab> gitlab # [ 282.758172] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:08.581Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"036ed70e4818580d9bd3e9ab","created_at":"2026-07-01T08:10:04.990Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9109f242c736e6c2c4cc086a6b0656d2","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.998Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-036ed70e4818580d9bd3e9ab: start","job_status":"start","queue_duration_s":3.583668,"scheduling_latency_s":3.583717}1533vm-test-run-nixbot-gitlab> gitlab # [ 282.875176] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:08.744Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"036ed70e4818580d9bd3e9ab","created_at":"2026-07-01T08:10:04.990Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9109f242c736e6c2c4cc086a6b0656d2","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:04.998Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-036ed70e4818580d9bd3e9ab: done: 0.162419 sec","job_status":"done","queue_duration_s":3.583668,"scheduling_latency_s":3.583717,"redis_calls":13,"redis_duration_s":0.031258,"redis_read_bytes":10,"redis_write_bytes":1343,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003946,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.013827,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.013485,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":368,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00344156,"extra.packages_cleanup_delete_orphaned_dependencies_worker.last_processed_packages_dependency_id":0,"extra.packages_cleanup_delete_orphaned_dependencies_worker.deleted_rows_count":0,"duration_s":0.162419,"completed_at":"2026-07-01T08:10:08.744Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.057553,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1534vm-test-run-nixbot-gitlab> gitlab # [ 282.968869] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:08.838Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"840e953ba91acb8c5cd3b676","created_at":"2026-07-01T08:10:05.008Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a27e2cbd72f0aa2e2820d2240cc2fc9","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:05.015Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-840e953ba91acb8c5cd3b676: start","job_status":"start","queue_duration_s":3.823317,"scheduling_latency_s":3.823367}1535vm-test-run-nixbot-gitlab> gitlab # [ 283.104625] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:08.973Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"840e953ba91acb8c5cd3b676","created_at":"2026-07-01T08:10:05.008Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a27e2cbd72f0aa2e2820d2240cc2fc9","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:05.015Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-840e953ba91acb8c5cd3b676: done: 0.135282 sec","job_status":"done","queue_duration_s":3.823317,"scheduling_latency_s":3.823367,"redis_calls":12,"redis_duration_s":0.018272,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003685,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007894,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006693,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00278474,"exclusive_lock_hold_duration_s":0.01421236,"duration_s":0.135282,"completed_at":"2026-07-01T08:10:08.973Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008253,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1536vm-test-run-nixbot-gitlab> gitlab # [ 283.211713] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.081Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"910a48186c766c752c64b8fb","created_at":"2026-07-01T08:10:05.021Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"583bb4c5bfe7ecba4266fec44052f938","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:05.025Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-910a48186c766c752c64b8fb: start","job_status":"start","queue_duration_s":4.055853,"scheduling_latency_s":4.055913}1537vm-test-run-nixbot-gitlab> gitlab # [ 283.375783] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.244Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"910a48186c766c752c64b8fb","created_at":"2026-07-01T08:10:05.021Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"583bb4c5bfe7ecba4266fec44052f938","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:05.025Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-910a48186c766c752c64b8fb: done: 0.163428 sec","job_status":"done","queue_duration_s":4.055853,"scheduling_latency_s":4.055913,"redis_calls":10,"redis_duration_s":0.013617,"redis_read_bytes":135,"redis_write_bytes":1302,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000609,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":133,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006634,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006374,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00246914,"exclusive_lock_hold_duration_s":0.03709914,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.163428,"completed_at":"2026-07-01T08:10:09.244Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.034248,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1538vm-test-run-nixbot-gitlab> gitlab # [ 283.473856] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.342Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"92021238d2e95528d9a913bd","created_at":"2026-07-01T08:10:05.041Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"baba84ef15bb53de8438c851d15b0764","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:05.065Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-92021238d2e95528d9a913bd: start","job_status":"start","queue_duration_s":4.277282,"scheduling_latency_s":4.277338}1539vm-test-run-nixbot-gitlab> gitlab # [ 283.623783] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.492Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"92021238d2e95528d9a913bd","created_at":"2026-07-01T08:10:05.041Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"baba84ef15bb53de8438c851d15b0764","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:05.065Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-92021238d2e95528d9a913bd: done: 0.150423 sec","job_status":"done","queue_duration_s":4.277282,"scheduling_latency_s":4.277338,"redis_calls":10,"redis_duration_s":0.027504,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.006379,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.016568,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004557,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033966,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00171112,"duration_s":0.150423,"completed_at":"2026-07-01T08:10:09.492Z","load_balancing_strategy":"primary","db_duration_s":0.018718,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1540vm-test-run-nixbot-gitlab> gitlab # [ 283.710143] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.579Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"c155432cfd74fdd23fc79ef3","created_at":"2026-07-01T08:10:05.085Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3b4c6a5858d302fbe3ac49ad448eaec4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-01T08:10:05.090Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-c155432cfd74fdd23fc79ef3: start","job_status":"start","queue_duration_s":4.488545,"scheduling_latency_s":4.488596}1541vm-test-run-nixbot-gitlab> gitlab # [ 283.783959] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.653Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1542vm-test-run-nixbot-gitlab> gitlab # [ 283.798134] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.667Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"c155432cfd74fdd23fc79ef3","created_at":"2026-07-01T08:10:05.085Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3b4c6a5858d302fbe3ac49ad448eaec4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-01T08:10:05.090Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-c155432cfd74fdd23fc79ef3: done: 0.087812 sec","job_status":"done","queue_duration_s":4.488545,"scheduling_latency_s":4.488596,"redis_calls":8,"redis_duration_s":0.006364,"redis_read_bytes":133,"redis_write_bytes":935,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000836,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":134,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00324,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002288,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.087812,"completed_at":"2026-07-01T08:10:09.667Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1543vm-test-run-nixbot-gitlab> gitlab # [ 283.884930] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.670Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"a35dd2963a5061e60ef3c54b","created_at":"2026-07-01T08:10:05.136Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20fabf8de41e217255c1e72a7c9914e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-01T08:10:05.155Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-a35dd2963a5061e60ef3c54b: start","job_status":"start","queue_duration_s":4.514858,"scheduling_latency_s":4.514908}1544vm-test-run-nixbot-gitlab> gitlab # [ 283.905526] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.691Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1545vm-test-run-nixbot-gitlab> gitlab # [ 283.910037] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.707Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"a35dd2963a5061e60ef3c54b","created_at":"2026-07-01T08:10:05.136Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20fabf8de41e217255c1e72a7c9914e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-01T08:10:05.155Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-a35dd2963a5061e60ef3c54b: done: 0.036352 sec","job_status":"done","queue_duration_s":4.514858,"scheduling_latency_s":4.514908,"redis_calls":8,"redis_duration_s":0.003233,"redis_read_bytes":133,"redis_write_bytes":941,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000348,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":135,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002047,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000838,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024533,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036352,"completed_at":"2026-07-01T08:10:09.706Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1546vm-test-run-nixbot-gitlab> gitlab # [ 283.955156] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.709Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"f8ae675b78b44f404bbfd1c2","created_at":"2026-07-01T08:10:05.189Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"455715b84357a78e1439b94e7f572c32","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:05.196Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-f8ae675b78b44f404bbfd1c2: start","job_status":"start","queue_duration_s":4.513188,"scheduling_latency_s":4.513225}1547vm-test-run-nixbot-gitlab> gitlab # [ 283.969807] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.796Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"f8ae675b78b44f404bbfd1c2","created_at":"2026-07-01T08:10:05.189Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"455715b84357a78e1439b94e7f572c32","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:05.196Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-f8ae675b78b44f404bbfd1c2: done: 0.086592 sec","job_status":"done","queue_duration_s":4.513188,"scheduling_latency_s":4.513225,"redis_calls":15,"redis_duration_s":0.00455,"redis_read_bytes":144,"redis_write_bytes":4147,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000239,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":117,"redis_queues_calls":2,"redis_queues_duration_s":0.000569,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002946,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000796,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.065325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.086592,"completed_at":"2026-07-01T08:10:09.796Z","load_balancing_strategy":"primary","db_duration_s":0.012958,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1548vm-test-run-nixbot-gitlab> gitlab # [ 284.012657] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.799Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"e8add654e9f0ce79fff07fd8","created_at":"2026-07-01T08:10:05.219Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82ef810b8e93568f2525c8d591012205","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:05.239Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-e8add654e9f0ce79fff07fd8: start","job_status":"start","queue_duration_s":4.559536,"scheduling_latency_s":4.559569}1549vm-test-run-nixbot-gitlab> gitlab # [ 284.027515] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.823Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"e8add654e9f0ce79fff07fd8","created_at":"2026-07-01T08:10:05.219Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-44f84741942b4305","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82ef810b8e93568f2525c8d591012205","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:05.239Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-e8add654e9f0ce79fff07fd8: done: 0.024051 sec","job_status":"done","queue_duration_s":4.559536,"scheduling_latency_s":4.559569,"redis_calls":9,"redis_duration_s":0.002556,"redis_read_bytes":261,"redis_write_bytes":1035,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000558,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":204,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001449,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000549,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015835,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024051,"completed_at":"2026-07-01T08:10:09.822Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1550vm-test-run-nixbot-gitlab> gitlab # [ 284.070544] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.825Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"1fa8e27f893444989f004b9c","created_at":"2026-07-01T08:10:07.154Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-4e4e11cfa3c640ca","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"93f1410c937bd1861384f1272ba469c8","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:07.183Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-1fa8e27f893444989f004b9c: start","job_status":"start","queue_duration_s":2.641869,"scheduling_latency_s":2.641899}1551vm-test-run-nixbot-gitlab> gitlab # [ 284.085792] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.870Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"1fa8e27f893444989f004b9c","created_at":"2026-07-01T08:10:07.154Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-4e4e11cfa3c640ca","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"93f1410c937bd1861384f1272ba469c8","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:07.183Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-1fa8e27f893444989f004b9c: done: 0.045223 sec","job_status":"done","queue_duration_s":2.641869,"scheduling_latency_s":2.641899,"redis_calls":10,"redis_duration_s":0.0031509999999999997,"redis_read_bytes":9,"redis_write_bytes":1117,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000607,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001956,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000588,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":222,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024318,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00051586,"duration_s":0.045223,"completed_at":"2026-07-01T08:10:09.870Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009845,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1552vm-test-run-nixbot-gitlab> gitlab # [ 284.129132] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.872Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"584d1bad681d25a226a5409e","created_at":"2026-07-01T08:10:07.187Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-4e4e11cfa3c640ca","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"93f1410c937bd1861384f1272ba469c8","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:07.209Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-584d1bad681d25a226a5409e: start","job_status":"start","queue_duration_s":2.663546,"scheduling_latency_s":2.663573}1553vm-test-run-nixbot-gitlab> gitlab # [ 284.144417] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.904Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"584d1bad681d25a226a5409e","created_at":"2026-07-01T08:10:07.187Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-4e4e11cfa3c640ca","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"93f1410c937bd1861384f1272ba469c8","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:07.209Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-584d1bad681d25a226a5409e: done: 0.031449 sec","job_status":"done","queue_duration_s":2.663546,"scheduling_latency_s":2.663573,"redis_calls":10,"redis_duration_s":0.002807,"redis_read_bytes":9,"redis_write_bytes":1133,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000577,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001676,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000554,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00044188,"duration_s":0.031449,"completed_at":"2026-07-01T08:10:09.904Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001508,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1554vm-test-run-nixbot-gitlab> gitlab # [ 284.189066] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8fe585274142c4b8f644dd98","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8fe585274142c4b8f644dd98: start","job_status":"start","queue_duration_s":1.624675,"scheduling_latency_s":1.624697}1555vm-test-run-nixbot-gitlab> gitlab # [ 284.201473] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.928Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8fe585274142c4b8f644dd98","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8fe585274142c4b8f644dd98: done: 0.022421 sec","job_status":"done","queue_duration_s":1.624675,"scheduling_latency_s":1.624697,"redis_calls":17,"redis_duration_s":0.0039900000000000005,"redis_read_bytes":522,"redis_write_bytes":1453,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001005,"redis_feature_flag_read_bytes":512,"redis_feature_flag_write_bytes":296,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001031,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014821,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022421,"completed_at":"2026-07-01T08:10:09.928Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1556vm-test-run-nixbot-gitlab> gitlab # [ 284.242666] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.930Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26af6495340df694581c47b8","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26af6495340df694581c47b8: start","job_status":"start","queue_duration_s":1.648558,"scheduling_latency_s":1.648575}1557vm-test-run-nixbot-gitlab> gitlab # [ 284.255544] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26af6495340df694581c47b8","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26af6495340df694581c47b8: done: 0.016443 sec","job_status":"done","queue_duration_s":1.648558,"scheduling_latency_s":1.648575,"redis_calls":13,"redis_duration_s":0.00275,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001779,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016443,"completed_at":"2026-07-01T08:10:09.946Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1558vm-test-run-nixbot-gitlab> gitlab # [ 284.295273] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f7ee7b0ad6da1c65e64722f","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f7ee7b0ad6da1c65e64722f: start","job_status":"start","queue_duration_s":1.66632,"scheduling_latency_s":1.666337}1559vm-test-run-nixbot-gitlab> gitlab # [ 284.307942] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.963Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f7ee7b0ad6da1c65e64722f","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f7ee7b0ad6da1c65e64722f: done: 0.015548 sec","job_status":"done","queue_duration_s":1.66632,"scheduling_latency_s":1.666337,"redis_calls":13,"redis_duration_s":0.002798,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000998,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015548,"completed_at":"2026-07-01T08:10:09.963Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1560vm-test-run-nixbot-gitlab> gitlab # [ 284.347775] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5d4da28c0e80ffbf4f06650","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5d4da28c0e80ffbf4f06650: start","job_status":"start","queue_duration_s":1.683417,"scheduling_latency_s":1.683434}1561vm-test-run-nixbot-gitlab> gitlab # [ 284.360592] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5d4da28c0e80ffbf4f06650","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5d4da28c0e80ffbf4f06650: done: 0.015105 sec","job_status":"done","queue_duration_s":1.683417,"scheduling_latency_s":1.683434,"redis_calls":13,"redis_duration_s":0.0026360000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000907,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001729,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009422,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015105,"completed_at":"2026-07-01T08:10:09.979Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1562vm-test-run-nixbot-gitlab> gitlab # [ 284.401150] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a822730da8e470063c86954f","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a822730da8e470063c86954f: start","job_status":"start","queue_duration_s":1.70003,"scheduling_latency_s":1.700047}1563vm-test-run-nixbot-gitlab> gitlab # [ 284.413753] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:09.998Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a822730da8e470063c86954f","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a822730da8e470063c86954f: done: 0.016903 sec","job_status":"done","queue_duration_s":1.70003,"scheduling_latency_s":1.700047,"redis_calls":13,"redis_duration_s":0.0033280000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000991,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002337,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016903,"completed_at":"2026-07-01T08:10:09.998Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1564vm-test-run-nixbot-gitlab> gitlab # [ 284.453468] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.000Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1d027b4a397e708930afb0a","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1d027b4a397e708930afb0a: start","job_status":"start","queue_duration_s":1.718579,"scheduling_latency_s":1.718597}1565vm-test-run-nixbot-gitlab> gitlab # [ 284.466142] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.016Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1d027b4a397e708930afb0a","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1d027b4a397e708930afb0a: done: 0.016005 sec","job_status":"done","queue_duration_s":1.718579,"scheduling_latency_s":1.718597,"redis_calls":13,"redis_duration_s":0.00301,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000994,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016005,"completed_at":"2026-07-01T08:10:10.015Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1566vm-test-run-nixbot-gitlab> gitlab # [ 284.506162] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4493abf4aa1d77ed31841a75","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4493abf4aa1d77ed31841a75: start","job_status":"start","queue_duration_s":1.736091,"scheduling_latency_s":1.736109}1567vm-test-run-nixbot-gitlab> gitlab # [ 284.518643] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4493abf4aa1d77ed31841a75","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4493abf4aa1d77ed31841a75: done: 0.015854 sec","job_status":"done","queue_duration_s":1.736091,"scheduling_latency_s":1.736109,"redis_calls":14,"redis_duration_s":0.002931,"redis_read_bytes":138,"redis_write_bytes":1242,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000208,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":85,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001794,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015854,"completed_at":"2026-07-01T08:10:10.033Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1568vm-test-run-nixbot-gitlab> gitlab # [ 284.559930] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.034Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4cf8e19975c4fc58dfa52842","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4cf8e19975c4fc58dfa52842: start","job_status":"start","queue_duration_s":1.753469,"scheduling_latency_s":1.753485}1569vm-test-run-nixbot-gitlab> gitlab # [ 284.572724] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.050Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4cf8e19975c4fc58dfa52842","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4cf8e19975c4fc58dfa52842: done: 0.015334 sec","job_status":"done","queue_duration_s":1.753469,"scheduling_latency_s":1.753485,"redis_calls":13,"redis_duration_s":0.002729,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001779,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015334,"completed_at":"2026-07-01T08:10:10.050Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1570vm-test-run-nixbot-gitlab> gitlab # [ 284.614237] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"675bbac728a90f8fa70781cb","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-675bbac728a90f8fa70781cb: start","job_status":"start","queue_duration_s":1.770391,"scheduling_latency_s":1.770408}1571vm-test-run-nixbot-gitlab> gitlab # [ 284.627004] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"675bbac728a90f8fa70781cb","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-675bbac728a90f8fa70781cb: done: 0.015327 sec","job_status":"done","queue_duration_s":1.770391,"scheduling_latency_s":1.770408,"redis_calls":13,"redis_duration_s":0.002696,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00176,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009756,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015327,"completed_at":"2026-07-01T08:10:10.067Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1572vm-test-run-nixbot-gitlab> gitlab # [ 284.666816] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09f6378a9659a748bb4a9d6c","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09f6378a9659a748bb4a9d6c: start","job_status":"start","queue_duration_s":1.787198,"scheduling_latency_s":1.787215}1573vm-test-run-nixbot-gitlab> gitlab # [ 284.679629] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09f6378a9659a748bb4a9d6c","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09f6378a9659a748bb4a9d6c: done: 0.015228 sec","job_status":"done","queue_duration_s":1.787198,"scheduling_latency_s":1.787215,"redis_calls":13,"redis_duration_s":0.002696,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001762,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015228,"completed_at":"2026-07-01T08:10:10.083Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1574vm-test-run-nixbot-gitlab> gitlab # [ 284.719330] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e3cfdc11996ab098ed564e8","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e3cfdc11996ab098ed564e8: start","job_status":"start","queue_duration_s":1.803745,"scheduling_latency_s":1.803761}1575vm-test-run-nixbot-gitlab> gitlab # [ 284.732134] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.105Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e3cfdc11996ab098ed564e8","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e3cfdc11996ab098ed564e8: done: 0.020115 sec","job_status":"done","queue_duration_s":1.803745,"scheduling_latency_s":1.803761,"redis_calls":13,"redis_duration_s":0.003027,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00115,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001877,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020115,"completed_at":"2026-07-01T08:10:10.105Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1576vm-test-run-nixbot-gitlab> gitlab # [ 284.771617] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.108Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7ab51027e87c6ace1b773cb","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7ab51027e87c6ace1b773cb: start","job_status":"start","queue_duration_s":1.827262,"scheduling_latency_s":1.827305}1577vm-test-run-nixbot-gitlab> gitlab # [ 284.785443] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7ab51027e87c6ace1b773cb","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7ab51027e87c6ace1b773cb: done: 0.037961 sec","job_status":"done","queue_duration_s":1.827262,"scheduling_latency_s":1.827305,"redis_calls":13,"redis_duration_s":0.0038720000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001381,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002491,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026414,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.037961,"completed_at":"2026-07-01T08:10:10.146Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1578vm-test-run-nixbot-gitlab> gitlab # [ 284.825153] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"828bce4c7256bda231821bc0","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-828bce4c7256bda231821bc0: start","job_status":"start","queue_duration_s":1.867359,"scheduling_latency_s":1.867392}1579vm-test-run-nixbot-gitlab> gitlab # [ 284.837583] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.174Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"828bce4c7256bda231821bc0","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-828bce4c7256bda231821bc0: done: 0.026013 sec","job_status":"done","queue_duration_s":1.867359,"scheduling_latency_s":1.867392,"redis_calls":13,"redis_duration_s":0.002539,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001667,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017259,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026013,"completed_at":"2026-07-01T08:10:10.174Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1580vm-test-run-nixbot-gitlab> gitlab # [ 284.877207] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.176Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"995b74eed333aa1c72cc905e","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-995b74eed333aa1c72cc905e: start","job_status":"start","queue_duration_s":1.894839,"scheduling_latency_s":1.894864}1581vm-test-run-nixbot-gitlab> gitlab # [ 284.889699] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.195Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"995b74eed333aa1c72cc905e","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-995b74eed333aa1c72cc905e: done: 0.019713 sec","job_status":"done","queue_duration_s":1.894839,"scheduling_latency_s":1.894864,"redis_calls":13,"redis_duration_s":0.001868,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00068,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001188,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013345,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019713,"completed_at":"2026-07-01T08:10:10.195Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1582vm-test-run-nixbot-gitlab> gitlab # [ 284.929425] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.197Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2aa5b457460abc9af56ab6f","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2aa5b457460abc9af56ab6f: start","job_status":"start","queue_duration_s":1.915985,"scheduling_latency_s":1.91601}1583vm-test-run-nixbot-gitlab> gitlab # [ 284.942240] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2aa5b457460abc9af56ab6f","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2aa5b457460abc9af56ab6f: done: 0.287044 sec","job_status":"done","queue_duration_s":1.915985,"scheduling_latency_s":1.91601,"redis_calls":13,"redis_duration_s":0.009955,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001208,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008747,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02934,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.287044,"completed_at":"2026-07-01T08:10:10.484Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1584vm-test-run-nixbot-gitlab> gitlab # [ 284.983148] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.487Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af446bcb35747a73772b0784","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af446bcb35747a73772b0784: start","job_status":"start","queue_duration_s":2.206114,"scheduling_latency_s":2.206164}1585vm-test-run-nixbot-gitlab> gitlab # [ 284.997106] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af446bcb35747a73772b0784","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af446bcb35747a73772b0784: done: 0.036733 sec","job_status":"done","queue_duration_s":2.206114,"scheduling_latency_s":2.206164,"redis_calls":13,"redis_duration_s":0.003604,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001115,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002489,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024217,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036733,"completed_at":"2026-07-01T08:10:10.524Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1586vm-test-run-nixbot-gitlab> gitlab # [ 285.036733] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.526Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"296e89320c25882c9c2cde04","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-296e89320c25882c9c2cde04: start","job_status":"start","queue_duration_s":2.244644,"scheduling_latency_s":2.244673}1587vm-test-run-nixbot-gitlab> gitlab # [ 285.049051] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.549Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"296e89320c25882c9c2cde04","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-296e89320c25882c9c2cde04: done: 0.023192 sec","job_status":"done","queue_duration_s":2.244644,"scheduling_latency_s":2.244673,"redis_calls":13,"redis_duration_s":0.002132,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000773,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001359,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023192,"completed_at":"2026-07-01T08:10:10.549Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1588vm-test-run-nixbot-gitlab> gitlab # [ 285.088761] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.550Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4110a401426ae124b22c997a","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4110a401426ae124b22c997a: start","job_status":"start","queue_duration_s":2.269287,"scheduling_latency_s":2.269311}1589vm-test-run-nixbot-gitlab> gitlab # [ 285.101225] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4110a401426ae124b22c997a","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4110a401426ae124b22c997a: done: 0.01857 sec","job_status":"done","queue_duration_s":2.269287,"scheduling_latency_s":2.269311,"redis_calls":13,"redis_duration_s":0.0017280000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000613,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001115,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011762,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01857,"completed_at":"2026-07-01T08:10:10.569Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1590vm-test-run-nixbot-gitlab> gitlab # [ 285.140597] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.571Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21792a3c11a507495a0eae0b","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21792a3c11a507495a0eae0b: start","job_status":"start","queue_duration_s":2.290192,"scheduling_latency_s":2.290214}1591vm-test-run-nixbot-gitlab> gitlab # [ 285.153169] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.588Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21792a3c11a507495a0eae0b","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21792a3c11a507495a0eae0b: done: 0.016515 sec","job_status":"done","queue_duration_s":2.290192,"scheduling_latency_s":2.290214,"redis_calls":13,"redis_duration_s":0.001578,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000606,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000972,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011615,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016515,"completed_at":"2026-07-01T08:10:10.588Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1592vm-test-run-nixbot-gitlab> gitlab # [ 285.193650] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.589Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccf0cd1fffd6d06a10fc1698","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccf0cd1fffd6d06a10fc1698: start","job_status":"start","queue_duration_s":2.307779,"scheduling_latency_s":2.307799}1593vm-test-run-nixbot-gitlab> gitlab # [ 285.206240] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.604Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccf0cd1fffd6d06a10fc1698","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccf0cd1fffd6d06a10fc1698: done: 0.015663 sec","job_status":"done","queue_duration_s":2.307779,"scheduling_latency_s":2.307799,"redis_calls":13,"redis_duration_s":0.0015010000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000987,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015663,"completed_at":"2026-07-01T08:10:10.604Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1594vm-test-run-nixbot-gitlab> gitlab # [ 285.245897] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.605Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce39daad2d107e1f87bb4035","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce39daad2d107e1f87bb4035: start","job_status":"start","queue_duration_s":2.324534,"scheduling_latency_s":2.324553}1595vm-test-run-nixbot-gitlab> gitlab # [ 285.258366] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.621Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce39daad2d107e1f87bb4035","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce39daad2d107e1f87bb4035: done: 0.015694 sec","job_status":"done","queue_duration_s":2.324534,"scheduling_latency_s":2.324553,"redis_calls":13,"redis_duration_s":0.0014490000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015694,"completed_at":"2026-07-01T08:10:10.621Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1596vm-test-run-nixbot-gitlab> gitlab # [ 285.297917] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.622Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9d84afb12ecb4a51c1d842b","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9d84afb12ecb4a51c1d842b: start","job_status":"start","queue_duration_s":2.341323,"scheduling_latency_s":2.341342}1597vm-test-run-nixbot-gitlab> gitlab # [ 285.310317] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9d84afb12ecb4a51c1d842b","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9d84afb12ecb4a51c1d842b: done: 0.016314 sec","job_status":"done","queue_duration_s":2.341323,"scheduling_latency_s":2.341342,"redis_calls":13,"redis_duration_s":0.001443,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016314,"completed_at":"2026-07-01T08:10:10.639Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1598vm-test-run-nixbot-gitlab> gitlab # [ 285.349859] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.640Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08fb3bc5c41157e8fed8b2a6","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08fb3bc5c41157e8fed8b2a6: start","job_status":"start","queue_duration_s":2.358712,"scheduling_latency_s":2.35873}1599vm-test-run-nixbot-gitlab> gitlab # [ 285.362277] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.655Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08fb3bc5c41157e8fed8b2a6","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08fb3bc5c41157e8fed8b2a6: done: 0.015854 sec","job_status":"done","queue_duration_s":2.358712,"scheduling_latency_s":2.35873,"redis_calls":13,"redis_duration_s":0.002468,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001951,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015854,"completed_at":"2026-07-01T08:10:10.655Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1600vm-test-run-nixbot-gitlab> gitlab # [ 285.402783] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.657Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ce015a83d4ee38419ea09b5","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ce015a83d4ee38419ea09b5: start","job_status":"start","queue_duration_s":2.375635,"scheduling_latency_s":2.375652}1601vm-test-run-nixbot-gitlab> gitlab # [ 285.415530] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.672Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ce015a83d4ee38419ea09b5","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ce015a83d4ee38419ea09b5: done: 0.015225 sec","job_status":"done","queue_duration_s":2.375635,"scheduling_latency_s":2.375652,"redis_calls":13,"redis_duration_s":0.0015119999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015225,"completed_at":"2026-07-01T08:10:10.672Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1602vm-test-run-nixbot-gitlab> gitlab # [ 285.455054] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e66a8295615e62cfe53507f8","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e66a8295615e62cfe53507f8: start","job_status":"start","queue_duration_s":2.391933,"scheduling_latency_s":2.391951}1603vm-test-run-nixbot-gitlab> gitlab # [ 285.467672] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e66a8295615e62cfe53507f8","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e66a8295615e62cfe53507f8: done: 0.015511 sec","job_status":"done","queue_duration_s":2.391933,"scheduling_latency_s":2.391951,"redis_calls":13,"redis_duration_s":0.001522,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009966,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015511,"completed_at":"2026-07-01T08:10:10.688Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1604vm-test-run-nixbot-gitlab> gitlab # [ 285.507653] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.691Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7b23d00550cb2874aaae0a7","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7b23d00550cb2874aaae0a7: start","job_status":"start","queue_duration_s":2.409734,"scheduling_latency_s":2.409753}1605vm-test-run-nixbot-gitlab> gitlab # [ 285.520423] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7b23d00550cb2874aaae0a7","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7b23d00550cb2874aaae0a7: done: 0.023183 sec","job_status":"done","queue_duration_s":2.409734,"scheduling_latency_s":2.409753,"redis_calls":13,"redis_duration_s":0.001931,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01669,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023183,"completed_at":"2026-07-01T08:10:10.714Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1606vm-test-run-nixbot-gitlab> gitlab # [ 285.560604] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.717Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e30b434a2c4c52751456d01","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e30b434a2c4c52751456d01: start","job_status":"start","queue_duration_s":2.436112,"scheduling_latency_s":2.436155}1607vm-test-run-nixbot-gitlab> gitlab # [ 285.573249] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e30b434a2c4c52751456d01","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e30b434a2c4c52751456d01: done: 0.042995 sec","job_status":"done","queue_duration_s":2.436112,"scheduling_latency_s":2.436155,"redis_calls":13,"redis_duration_s":0.004782,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001447,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003335,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.042995,"completed_at":"2026-07-01T08:10:10.760Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1608vm-test-run-nixbot-gitlab> gitlab # [ 285.614550] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.763Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a458a3ea1b6318b97a87f6d0","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a458a3ea1b6318b97a87f6d0: start","job_status":"start","queue_duration_s":2.481573,"scheduling_latency_s":2.48161}1609vm-test-run-nixbot-gitlab> gitlab # [ 285.627265] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a458a3ea1b6318b97a87f6d0","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a458a3ea1b6318b97a87f6d0: done: 0.027266 sec","job_status":"done","queue_duration_s":2.481573,"scheduling_latency_s":2.48161,"redis_calls":13,"redis_duration_s":0.002461,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000856,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001605,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018725,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027266,"completed_at":"2026-07-01T08:10:10.790Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1610vm-test-run-nixbot-gitlab> gitlab # [ 285.667718] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fcdb5a382bad822aeb5b27c","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fcdb5a382bad822aeb5b27c: start","job_status":"start","queue_duration_s":2.51034,"scheduling_latency_s":2.510365}1611vm-test-run-nixbot-gitlab> gitlab # [ 285.680706] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.814Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fcdb5a382bad822aeb5b27c","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fcdb5a382bad822aeb5b27c: done: 0.023044 sec","job_status":"done","queue_duration_s":2.51034,"scheduling_latency_s":2.510365,"redis_calls":13,"redis_duration_s":0.0035800000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002609,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015208,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023044,"completed_at":"2026-07-01T08:10:10.814Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1612vm-test-run-nixbot-gitlab> gitlab # [ 285.721159] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70aadc36d563c1d21cca4d6b","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70aadc36d563c1d21cca4d6b: start","job_status":"start","queue_duration_s":2.534633,"scheduling_latency_s":2.534652}1613vm-test-run-nixbot-gitlab> gitlab # [ 285.733769] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70aadc36d563c1d21cca4d6b","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70aadc36d563c1d21cca4d6b: done: 0.016829 sec","job_status":"done","queue_duration_s":2.534633,"scheduling_latency_s":2.534652,"redis_calls":13,"redis_duration_s":0.0015890000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000568,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016829,"completed_at":"2026-07-01T08:10:10.832Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1614vm-test-run-nixbot-gitlab> gitlab # [ 285.773916] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.834Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5647ef012bb5556e8d2bc0e1","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5647ef012bb5556e8d2bc0e1: start","job_status":"start","queue_duration_s":2.552627,"scheduling_latency_s":2.55266}1615vm-test-run-nixbot-gitlab> gitlab # [ 285.786569] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5647ef012bb5556e8d2bc0e1","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5647ef012bb5556e8d2bc0e1: done: 0.015505 sec","job_status":"done","queue_duration_s":2.552627,"scheduling_latency_s":2.55266,"redis_calls":13,"redis_duration_s":0.0014299999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000918,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015505,"completed_at":"2026-07-01T08:10:10.849Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1616vm-test-run-nixbot-gitlab> gitlab # [ 285.828621] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.851Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21cde762c829253370bf4133","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21cde762c829253370bf4133: start","job_status":"start","queue_duration_s":2.570329,"scheduling_latency_s":2.570347}1617vm-test-run-nixbot-gitlab> gitlab # [ 285.866543] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21cde762c829253370bf4133","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21cde762c829253370bf4133: done: 0.015642 sec","job_status":"done","queue_duration_s":2.570329,"scheduling_latency_s":2.570347,"redis_calls":13,"redis_duration_s":0.001505,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000586,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015642,"completed_at":"2026-07-01T08:10:10.867Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1618vm-test-run-nixbot-gitlab> gitlab # [ 285.937415] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c917c7d9b4e6c34bfb69b61","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c917c7d9b4e6c34bfb69b61: start","job_status":"start","queue_duration_s":2.586999,"scheduling_latency_s":2.587015}1619vm-test-run-nixbot-gitlab> gitlab # [ 285.951916] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c917c7d9b4e6c34bfb69b61","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c917c7d9b4e6c34bfb69b61: done: 0.015697 sec","job_status":"done","queue_duration_s":2.586999,"scheduling_latency_s":2.587015,"redis_calls":13,"redis_duration_s":0.0014320000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011312,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015697,"completed_at":"2026-07-01T08:10:10.884Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1620vm-test-run-nixbot-gitlab> gitlab # [ 285.993286] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.885Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab8a9162a296d3e9de493917","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab8a9162a296d3e9de493917: start","job_status":"start","queue_duration_s":2.603742,"scheduling_latency_s":2.603758}1621vm-test-run-nixbot-gitlab> gitlab # [ 286.005999] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab8a9162a296d3e9de493917","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab8a9162a296d3e9de493917: done: 0.015533 sec","job_status":"done","queue_duration_s":2.603742,"scheduling_latency_s":2.603758,"redis_calls":13,"redis_duration_s":0.00144,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015533,"completed_at":"2026-07-01T08:10:10.900Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1622vm-test-run-nixbot-gitlab> gitlab # [ 286.047424] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c99d02859ec06fe1fd385a1","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c99d02859ec06fe1fd385a1: start","job_status":"start","queue_duration_s":2.621656,"scheduling_latency_s":2.621675}1623vm-test-run-nixbot-gitlab> gitlab # [ 286.060109] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c99d02859ec06fe1fd385a1","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c99d02859ec06fe1fd385a1: done: 0.015893 sec","job_status":"done","queue_duration_s":2.621656,"scheduling_latency_s":2.621675,"redis_calls":13,"redis_duration_s":0.002476,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001401,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001075,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015893,"completed_at":"2026-07-01T08:10:10.918Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1624vm-test-run-nixbot-gitlab> gitlab # [ 286.099810] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89dbaa2496ea1b51ff3fd31e","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89dbaa2496ea1b51ff3fd31e: start","job_status":"start","queue_duration_s":2.638564,"scheduling_latency_s":2.638581}1625vm-test-run-nixbot-gitlab> gitlab # [ 286.112675] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.937Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89dbaa2496ea1b51ff3fd31e","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89dbaa2496ea1b51ff3fd31e: done: 0.017299 sec","job_status":"done","queue_duration_s":2.638564,"scheduling_latency_s":2.638581,"redis_calls":13,"redis_duration_s":0.00147,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017299,"completed_at":"2026-07-01T08:10:10.937Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1626vm-test-run-nixbot-gitlab> gitlab # [ 286.152331] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.938Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e88618c94d65914e345feb6","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e88618c94d65914e345feb6: start","job_status":"start","queue_duration_s":2.65696,"scheduling_latency_s":2.656977}1627vm-test-run-nixbot-gitlab> gitlab # [ 286.164877] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.953Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e88618c94d65914e345feb6","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e88618c94d65914e345feb6: done: 0.015369 sec","job_status":"done","queue_duration_s":2.65696,"scheduling_latency_s":2.656977,"redis_calls":13,"redis_duration_s":0.001411,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015369,"completed_at":"2026-07-01T08:10:10.953Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1628vm-test-run-nixbot-gitlab> gitlab # [ 286.204807] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a35e672a697c49d2fcc4693c","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a35e672a697c49d2fcc4693c: start","job_status":"start","queue_duration_s":2.674005,"scheduling_latency_s":2.674023}1629vm-test-run-nixbot-gitlab> gitlab # [ 286.218056] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.969Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a35e672a697c49d2fcc4693c","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a35e672a697c49d2fcc4693c: done: 0.014356 sec","job_status":"done","queue_duration_s":2.674005,"scheduling_latency_s":2.674023,"redis_calls":13,"redis_duration_s":0.0014,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000884,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014356,"completed_at":"2026-07-01T08:10:10.969Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1630vm-test-run-nixbot-gitlab> gitlab # [ 286.257834] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70d0b5160949b0a83b4bb109","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70d0b5160949b0a83b4bb109: start","job_status":"start","queue_duration_s":2.689417,"scheduling_latency_s":2.689433}1631vm-test-run-nixbot-gitlab> gitlab # [ 286.270359] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.985Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70d0b5160949b0a83b4bb109","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70d0b5160949b0a83b4bb109: done: 0.01447 sec","job_status":"done","queue_duration_s":2.689417,"scheduling_latency_s":2.689433,"redis_calls":13,"redis_duration_s":0.00147,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01447,"completed_at":"2026-07-01T08:10:10.985Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1632vm-test-run-nixbot-gitlab> gitlab # [ 286.310127] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:10.986Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72f324798d6bff307321ae08","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72f324798d6bff307321ae08: start","job_status":"start","queue_duration_s":2.704937,"scheduling_latency_s":2.704956}1633vm-test-run-nixbot-gitlab> gitlab # [ 286.322975] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.001Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72f324798d6bff307321ae08","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72f324798d6bff307321ae08: done: 0.014756 sec","job_status":"done","queue_duration_s":2.704937,"scheduling_latency_s":2.704956,"redis_calls":13,"redis_duration_s":0.00141,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014756,"completed_at":"2026-07-01T08:10:11.001Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1634vm-test-run-nixbot-gitlab> gitlab # [ 286.362768] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.002Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a635696b6a209798caff579b","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a635696b6a209798caff579b: start","job_status":"start","queue_duration_s":2.720704,"scheduling_latency_s":2.720722}1635vm-test-run-nixbot-gitlab> gitlab # [ 286.375558] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a635696b6a209798caff579b","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a635696b6a209798caff579b: done: 0.015232 sec","job_status":"done","queue_duration_s":2.720704,"scheduling_latency_s":2.720722,"redis_calls":13,"redis_duration_s":0.001678,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015232,"completed_at":"2026-07-01T08:10:11.017Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1636vm-test-run-nixbot-gitlab> gitlab # [ 286.415512] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.018Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f35205f3dc57f1530ef4a3ee","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f35205f3dc57f1530ef4a3ee: start","job_status":"start","queue_duration_s":2.737069,"scheduling_latency_s":2.737087}1637vm-test-run-nixbot-gitlab> gitlab # [ 286.429007] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f35205f3dc57f1530ef4a3ee","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f35205f3dc57f1530ef4a3ee: done: 0.014456 sec","job_status":"done","queue_duration_s":2.737069,"scheduling_latency_s":2.737087,"redis_calls":13,"redis_duration_s":0.001424,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010208,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014456,"completed_at":"2026-07-01T08:10:11.032Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1638vm-test-run-nixbot-gitlab> gitlab # [ 286.470578] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.034Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95bfc80b8d6b0e03becd9b6d","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95bfc80b8d6b0e03becd9b6d: start","job_status":"start","queue_duration_s":2.752563,"scheduling_latency_s":2.75258}1639vm-test-run-nixbot-gitlab> gitlab # [ 286.483490] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.048Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95bfc80b8d6b0e03becd9b6d","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95bfc80b8d6b0e03becd9b6d: done: 0.014261 sec","job_status":"done","queue_duration_s":2.752563,"scheduling_latency_s":2.75258,"redis_calls":13,"redis_duration_s":0.001398,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000897,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014261,"completed_at":"2026-07-01T08:10:11.048Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1640vm-test-run-nixbot-gitlab> gitlab # [ 286.524535] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.049Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb84a49e946c62738d9186f9","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb84a49e946c62738d9186f9: start","job_status":"start","queue_duration_s":2.767826,"scheduling_latency_s":2.767841}1641vm-test-run-nixbot-gitlab> gitlab # [ 286.537070] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.064Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb84a49e946c62738d9186f9","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb84a49e946c62738d9186f9: done: 0.01537 sec","job_status":"done","queue_duration_s":2.767826,"scheduling_latency_s":2.767841,"redis_calls":13,"redis_duration_s":0.0014629999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01537,"completed_at":"2026-07-01T08:10:11.064Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1642vm-test-run-nixbot-gitlab> gitlab # [ 286.577380] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7494d31855b1864744f2b496","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7494d31855b1864744f2b496: start","job_status":"start","queue_duration_s":2.78426,"scheduling_latency_s":2.784277}1643vm-test-run-nixbot-gitlab> gitlab # [ 286.590166] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.079Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7494d31855b1864744f2b496","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7494d31855b1864744f2b496: done: 0.014209 sec","job_status":"done","queue_duration_s":2.78426,"scheduling_latency_s":2.784277,"redis_calls":13,"redis_duration_s":0.0014060000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000903,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014209,"completed_at":"2026-07-01T08:10:11.079Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1644vm-test-run-nixbot-gitlab> gitlab # [ 286.631539] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.080Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96581792739fc0e3554fbbb7","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96581792739fc0e3554fbbb7: start","job_status":"start","queue_duration_s":2.799506,"scheduling_latency_s":2.799522}1645vm-test-run-nixbot-gitlab> gitlab # [ 286.644654] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.095Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96581792739fc0e3554fbbb7","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96581792739fc0e3554fbbb7: done: 0.014134 sec","job_status":"done","queue_duration_s":2.799506,"scheduling_latency_s":2.799522,"redis_calls":13,"redis_duration_s":0.0014039999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000911,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014134,"completed_at":"2026-07-01T08:10:11.095Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1646vm-test-run-nixbot-gitlab> gitlab # [ 286.685164] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.096Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"816107b4d17dc23b7959d71c","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-816107b4d17dc23b7959d71c: start","job_status":"start","queue_duration_s":2.815104,"scheduling_latency_s":2.815121}1647vm-test-run-nixbot-gitlab> gitlab # [ 286.698357] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.111Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"816107b4d17dc23b7959d71c","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-816107b4d17dc23b7959d71c: done: 0.014579 sec","job_status":"done","queue_duration_s":2.815104,"scheduling_latency_s":2.815121,"redis_calls":13,"redis_duration_s":0.0015170000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000973,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014579,"completed_at":"2026-07-01T08:10:11.111Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1648vm-test-run-nixbot-gitlab> gitlab # [ 286.738760] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.112Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6842c21af35300f43d98fa3","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6842c21af35300f43d98fa3: start","job_status":"start","queue_duration_s":2.830685,"scheduling_latency_s":2.830701}1649vm-test-run-nixbot-gitlab> gitlab # [ 286.751415] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6842c21af35300f43d98fa3","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6842c21af35300f43d98fa3: done: 0.014539 sec","job_status":"done","queue_duration_s":2.830685,"scheduling_latency_s":2.830701,"redis_calls":13,"redis_duration_s":0.0014130000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000903,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014539,"completed_at":"2026-07-01T08:10:11.126Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1650vm-test-run-nixbot-gitlab> gitlab # [ 286.792152] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33e76077bad3fc70ab6b09b0","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33e76077bad3fc70ab6b09b0: start","job_status":"start","queue_duration_s":2.846257,"scheduling_latency_s":2.846274}1651vm-test-run-nixbot-gitlab> gitlab # [ 286.804889] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33e76077bad3fc70ab6b09b0","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33e76077bad3fc70ab6b09b0: done: 0.014483 sec","job_status":"done","queue_duration_s":2.846257,"scheduling_latency_s":2.846274,"redis_calls":13,"redis_duration_s":0.001398,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010248,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014483,"completed_at":"2026-07-01T08:10:11.142Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1652vm-test-run-nixbot-gitlab> gitlab # [ 286.845807] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.143Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8935d66fda18402ee3ef2db1","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8935d66fda18402ee3ef2db1: start","job_status":"start","queue_duration_s":2.862381,"scheduling_latency_s":2.862399}1653vm-test-run-nixbot-gitlab> gitlab # [ 286.858584] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.158Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8935d66fda18402ee3ef2db1","created_at":"2026-07-01T08:10:07.865Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-b77c5810eea64da0","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1753ede03727cf511ce317847ca27be1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:08.281Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8935d66fda18402ee3ef2db1: done: 0.014398 sec","job_status":"done","queue_duration_s":2.862381,"scheduling_latency_s":2.862399,"redis_calls":13,"redis_duration_s":0.0018419999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014398,"completed_at":"2026-07-01T08:10:11.158Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1654vm-test-run-nixbot-gitlab> gitlab # [ 286.899059] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.160Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7cddf006d5e2e6858a482e5c","created_at":"2026-07-01T08:10:09.729Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-c188db9d7d3c4fa5","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"455715b84357a78e1439b94e7f572c32","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:09.786Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7cddf006d5e2e6858a482e5c: start","job_status":"start","queue_duration_s":1.37414,"scheduling_latency_s":1.374187}1655vm-test-run-nixbot-gitlab> gitlab # [ 286.911900] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.177Z","message":"No existing merge request to be cleaned up."}1656vm-test-run-nixbot-gitlab> gitlab # [ 286.913746] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.185Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7cddf006d5e2e6858a482e5c","created_at":"2026-07-01T08:10:09.729Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-c188db9d7d3c4fa5","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"455715b84357a78e1439b94e7f572c32","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:09.786Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7cddf006d5e2e6858a482e5c: done: 0.024683 sec","job_status":"done","queue_duration_s":1.37414,"scheduling_latency_s":1.374187,"redis_calls":12,"redis_duration_s":0.001979,"redis_read_bytes":264,"redis_write_bytes":1213,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000267,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":191,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001147,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024683,"completed_at":"2026-07-01T08:10:11.185Z","load_balancing_strategy":"primary","db_duration_s":0.003359,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1657vm-test-run-nixbot-gitlab> gitlab # [ 286.954859] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.186Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"af575f2fb9b9a71c6b1b1660","created_at":"2026-07-01T08:10:09.729Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-c188db9d7d3c4fa5","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"455715b84357a78e1439b94e7f572c32","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:09.786Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-af575f2fb9b9a71c6b1b1660: start","job_status":"start","queue_duration_s":1.399962,"scheduling_latency_s":1.399987}1658vm-test-run-nixbot-gitlab> gitlab # [ 286.967688] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.198Z","message":"No existing merge request to be cleaned up."}1659vm-test-run-nixbot-gitlab> gitlab # [ 286.969500] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.205Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"af575f2fb9b9a71c6b1b1660","created_at":"2026-07-01T08:10:09.729Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-c188db9d7d3c4fa5","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"455715b84357a78e1439b94e7f572c32","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:09.786Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-af575f2fb9b9a71c6b1b1660: done: 0.018974 sec","job_status":"done","queue_duration_s":1.399962,"scheduling_latency_s":1.399987,"redis_calls":10,"redis_duration_s":0.001301,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000759,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018974,"completed_at":"2026-07-01T08:10:11.205Z","load_balancing_strategy":"primary","db_duration_s":0.002603,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1660vm-test-run-nixbot-gitlab> gitlab # [ 287.008764] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.207Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9d450d2fb27d138fa1f17d4f","created_at":"2026-07-01T08:10:09.729Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-c188db9d7d3c4fa5","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"455715b84357a78e1439b94e7f572c32","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:09.786Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9d450d2fb27d138fa1f17d4f: start","job_status":"start","queue_duration_s":1.4205,"scheduling_latency_s":1.420517}1661vm-test-run-nixbot-gitlab> gitlab # [ 287.021278] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.217Z","message":"No existing merge request to be cleaned up."}1662vm-test-run-nixbot-gitlab> gitlab # [ 287.023006] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.225Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9d450d2fb27d138fa1f17d4f","created_at":"2026-07-01T08:10:09.729Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-c188db9d7d3c4fa5","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"455715b84357a78e1439b94e7f572c32","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:09.786Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9d450d2fb27d138fa1f17d4f: done: 0.018172 sec","job_status":"done","queue_duration_s":1.4205,"scheduling_latency_s":1.420517,"redis_calls":10,"redis_duration_s":0.001473,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000925,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018172,"completed_at":"2026-07-01T08:10:11.225Z","load_balancing_strategy":"primary","db_duration_s":0.002411,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1663vm-test-run-nixbot-gitlab> gitlab # [ 287.063920] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.226Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c1819a53f8e8ecb8578d220d","created_at":"2026-07-01T08:10:09.729Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-c188db9d7d3c4fa5","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"455715b84357a78e1439b94e7f572c32","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:09.786Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c1819a53f8e8ecb8578d220d: start","job_status":"start","queue_duration_s":1.439714,"scheduling_latency_s":1.43973}1664vm-test-run-nixbot-gitlab> gitlab # [ 287.076935] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.237Z","message":"No existing merge request to be cleaned up."}1665vm-test-run-nixbot-gitlab> gitlab # [ 287.078629] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:11.246Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c1819a53f8e8ecb8578d220d","created_at":"2026-07-01T08:10:09.729Z","trace_propagation_headers":{"sentry-trace":"6e591f3d7cd64fe3a8ed5e96b6f4584d-c188db9d7d3c4fa5","baggage":"sentry-trace_id=6e591f3d7cd64fe3a8ed5e96b6f4584d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"455715b84357a78e1439b94e7f572c32","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:09.786Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c1819a53f8e8ecb8578d220d: done: 0.020163 sec","job_status":"done","queue_duration_s":1.439714,"scheduling_latency_s":1.43973,"redis_calls":10,"redis_duration_s":0.001849,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00082,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001029,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020163,"completed_at":"2026-07-01T08:10:11.246Z","load_balancing_strategy":"primary","db_duration_s":0.002427,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1666vm-test-run-nixbot-gitlab> gitlab # [ 294.766158] runuser[3303]: pam_unix(runuser:session): session closed for user gitlab1667vm-test-run-nixbot-gitlab> gitlab # [ 294.771707] setup-gitlab-start[3634]: ++ jq -r .token1668vm-test-run-nixbot-gitlab> gitlab # [ 294.774672] setup-gitlab-start[3635]: +++ date -d '+7 days' +%F1669vm-test-run-nixbot-gitlab> gitlab # [ 294.778165] setup-gitlab-start[3633]: ++ curl -fs -X POST -H 'Authorization: Bearer cea197d264c416dae79fcf35f3023b1b4d35e93ae52ee2d457fc35f43b40961a' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-08"}' http://gitlab/api/v4/users/1/personal_access_tokens1670vm-test-run-nixbot-gitlab> gitlab # [ 295.175546] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:21.043Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"837e225def0cf67d14c69bdb","created_at":"2026-07-01T08:10:21.005Z","trace_propagation_headers":{"sentry-trace":"74ea03d6bcc14fd584618d0c696b0e22-24750798b8124e38","baggage":"sentry-trace_id=74ea03d6bcc14fd584618d0c696b0e22,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBN83XQSK2C1A3KP68TV98","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:8a0b0bfa5e3ed36f082ab515f0a6a915532f69b11a7ea894bec2a01ff0b9df38","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:21.038Z","job_size_bytes":451,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-837e225def0cf67d14c69bdb: start","job_status":"start","queue_duration_s":0.005267,"scheduling_latency_s":0.005334}1671vm-test-run-nixbot-gitlab> gitlab # [ 295.229508] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:21 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 3841672vm-test-run-nixbot-gitlab> gitlab # [ 295.235069] setup-gitlab-start[3256]: + TOKEN=glpat-cfCOHVx5pibUnm7_o_gmTm86MQp1OjEH.01.0w1hnds5f1673vm-test-run-nixbot-gitlab> gitlab # [ 295.237740] setup-gitlab-start[3256]: + test -n glpat-cfCOHVx5pibUnm7_o_gmTm86MQp1OjEH.01.0w1hnds5f1674vm-test-run-nixbot-gitlab> gitlab # [ 295.240320] setup-gitlab-start[3256]: + test glpat-cfCOHVx5pibUnm7_o_gmTm86MQp1OjEH.01.0w1hnds5f '!=' null1675vm-test-run-nixbot-gitlab> gitlab # [ 295.242912] setup-gitlab-start[3256]: + mkdir -p /var/lib/secrets1676vm-test-run-nixbot-gitlab> gitlab # [ 295.244581] setup-gitlab-start[3256]: + echo glpat-cfCOHVx5pibUnm7_o_gmTm86MQp1OjEH.01.0w1hnds5f1677vm-test-run-nixbot-gitlab> gitlab # [ 295.246805] setup-gitlab-start[3256]: + chmod 644 /var/lib/secrets/gitlab-token1678vm-test-run-nixbot-gitlab> gitlab # [ 295.248658] setup-gitlab-start[3256]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-cfCOHVx5pibUnm7_o_gmTm86MQp1OjEH.01.0w1hnds5f' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1679vm-test-run-nixbot-gitlab> gitlab # [ 295.459898] sidekiq-cluster[2429]: {"severity":"WARN","time":"2026-07-01T08:10:21.328Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"837e225def0cf67d14c69bdb","created_at":"2026-07-01T08:10:21.005Z","trace_propagation_headers":{"sentry-trace":"74ea03d6bcc14fd584618d0c696b0e22-24750798b8124e38","baggage":"sentry-trace_id=74ea03d6bcc14fd584618d0c696b0e22,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBN83XQSK2C1A3KP68TV98","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:8a0b0bfa5e3ed36f082ab515f0a6a915532f69b11a7ea894bec2a01ff0b9df38","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:21.038Z","job_size_bytes":451,"pid":2429,"sidekiq_tid":"lylh","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-837e225def0cf67d14c69bdb: fail: 0.284285 sec","job_status":"fail","queue_duration_s":0.005267,"scheduling_latency_s":0.005334,"redis_calls":11,"redis_duration_s":0.005863,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001895,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003003,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.217187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00073112,"duration_s":0.284285,"completed_at":"2026-07-01T08:10:21.327Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/w9mp1gyfs2qdv12r0ji70idzndmxwz2s-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/p8ylyr0sj9sxcknqcpja13nypqixk2dp-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/dndr7b4a17339dsm67vzb6nv4p2kp82q-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/zx6ndpscxs64hnidz69ab430ccms6rak-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.046033,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1680vm-test-run-nixbot-gitlab> gitlab # [ 296.607788] gitaly[1368]: time="2026-07-01T08:10:22.476Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBN8GNCYNWC1S02XVVB39P error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T08:10:32.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:22.476" pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1681vm-test-run-nixbot-gitlab> gitlab # [ 296.638576] gitaly[1368]: time="2026-07-01T08:10:22.479Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBN8GNCYNWC1S02XVVB39P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T08:10:32.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:22.476" grpc.time_ms=2.272 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1682vm-test-run-nixbot-gitlab> gitlab # [ 297.093365] gitaly[1368]: time="2026-07-01T08:10:22.963Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBN8GNCYNWC1S02XVVB39P error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T08:10:32.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:22.963" pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1683vm-test-run-nixbot-gitlab> gitlab # [ 297.124645] gitaly[1368]: time="2026-07-01T08:10:22.963Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBN8GNCYNWC1S02XVVB39P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T08:10:32.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:22.963" grpc.time_ms=0.573 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1684vm-test-run-nixbot-gitlab> gitlab # [ 297.160175] gitaly[1368]: time="2026-07-01T08:10:23.017Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBN8GNCYNWC1S02XVVB39P diskcache=5616898a-b053-4eea-814f-c5253cfc64a3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-01T08:10:32.026" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:22.987" pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1685vm-test-run-nixbot-gitlab> gitlab # [ 297.185491] gitaly[1368]: time="2026-07-01T08:10:23.018Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=298 command.oublock=32 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBN8GNCYNWC1S02XVVB39P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-01T08:10:32.026" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:22.987" grpc.time_ms=30.483 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1686vm-test-run-nixbot-gitlab> gitlab # [ 297.209125] gitaly[1368]: time="2026-07-01T08:10:23.074Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBN8GNCYNWC1S02XVVB39P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T08:10:33.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:23.072" pid=1368 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1687vm-test-run-nixbot-gitlab> gitlab # [ 297.223678] gitaly[1368]: time="2026-07-01T08:10:23.074Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBN8GNCYNWC1S02XVVB39P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T08:10:33.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:23.072" grpc.time_ms=1.402 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1688vm-test-run-nixbot-gitlab> gitlab # [ 297.237389] gitaly[1368]: time="2026-07-01T08:10:23.106Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBN8GNCYNWC1S02XVVB39P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-01T08:10:33.018" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:23.088" grpc.time_ms=16.933 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1689vm-test-run-nixbot-gitlab> gitlab # [ 297.321459] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:23.190Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"e73faf5e2471d2a6c8b054b6","created_at":"2026-07-01T08:10:23.129Z","trace_propagation_headers":{"sentry-trace":"73117be61270440b93728abe508314ed-db44862417984ffc","baggage":"sentry-trace_id=73117be61270440b93728abe508314ed,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBN8GNCYNWC1S02XVVB39P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:23.187Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-e73faf5e2471d2a6c8b054b6: start","job_status":"start","queue_duration_s":0.001987,"scheduling_latency_s":0.002037}1690vm-test-run-nixbot-gitlab> gitlab # [ 297.408481] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:23.277Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"e73faf5e2471d2a6c8b054b6","created_at":"2026-07-01T08:10:23.129Z","trace_propagation_headers":{"sentry-trace":"73117be61270440b93728abe508314ed-db44862417984ffc","baggage":"sentry-trace_id=73117be61270440b93728abe508314ed,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBN8GNCYNWC1S02XVVB39P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:23.187Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-e73faf5e2471d2a6c8b054b6: done: 0.088051 sec","job_status":"done","queue_duration_s":0.001987,"scheduling_latency_s":0.002037,"redis_calls":13,"redis_duration_s":0.004287,"redis_read_bytes":138,"redis_write_bytes":1593,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00094,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":428,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001892,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001455,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.063313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.04890272,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.088051,"completed_at":"2026-07-01T08:10:23.277Z","load_balancing_strategy":"primary","db_duration_s":0.015843,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1691vm-test-run-nixbot-gitlab> gitlab # [ 297.512196] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:23.381Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"155c46b57686ffbd83b0d6f0","created_at":"2026-07-01T08:10:23.330Z","trace_propagation_headers":{"sentry-trace":"73117be61270440b93728abe508314ed-db44862417984ffc","baggage":"sentry-trace_id=73117be61270440b93728abe508314ed,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBN8GNCYNWC1S02XVVB39P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:23.380Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-155c46b57686ffbd83b0d6f0: start","job_status":"start","queue_duration_s":0.001516,"scheduling_latency_s":0.00155}1692vm-test-run-nixbot-gitlab> gitlab # [ 297.717198] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:23.586Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"155c46b57686ffbd83b0d6f0","created_at":"2026-07-01T08:10:23.330Z","trace_propagation_headers":{"sentry-trace":"73117be61270440b93728abe508314ed-db44862417984ffc","baggage":"sentry-trace_id=73117be61270440b93728abe508314ed,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBN8GNCYNWC1S02XVVB39P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:23.380Z","job_size_bytes":2,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-155c46b57686ffbd83b0d6f0: done: 0.204454 sec","job_status":"done","queue_duration_s":0.001516,"scheduling_latency_s":0.00155,"redis_calls":14,"redis_duration_s":0.006382,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001834,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002838,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00171,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.146104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.15119896000000002,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.204454,"completed_at":"2026-07-01T08:10:23.586Z","load_balancing_strategy":"primary","db_duration_s":0.046021,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1693vm-test-run-nixbot-gitlab> gitlab # [ 298.188418] gitaly[1368]: time="2026-07-01T08:10:24.057Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBN8GNCYNWC1S02XVVB39P error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T08:10:34.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:24.056" pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1694vm-test-run-nixbot-gitlab> gitlab # [ 298.222370] gitaly[1368]: time="2026-07-01T08:10:24.057Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBN8GNCYNWC1S02XVVB39P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T08:10:34.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:24.056" grpc.time_ms=0.77 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1695vm-test-run-nixbot-gitlab> gitlab # [ 298.253323] gitaly[1368]: time="2026-07-01T08:10:24.090Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBN8GNCYNWC1S02XVVB39P diskcache=3aee1cdc-01c8-47a3-be19-413a528a0c9a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-01T08:10:34.010" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:24.060" pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1696vm-test-run-nixbot-gitlab> gitlab # [ 298.274687] gitaly[1368]: time="2026-07-01T08:10:24.091Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=299 command.oublock=32 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBN8GNCYNWC1S02XVVB39P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-01T08:10:34.010" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:24.060" grpc.time_ms=30.216 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1697vm-test-run-nixbot-gitlab> gitlab # [ 298.296844] gitaly[1368]: time="2026-07-01T08:10:24.098Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBN8GNCYNWC1S02XVVB39P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T08:10:34.026" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:24.097" pid=1368 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1698vm-test-run-nixbot-gitlab> gitlab # [ 298.314714] gitaly[1368]: time="2026-07-01T08:10:24.098Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBN8GNCYNWC1S02XVVB39P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T08:10:34.026" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:24.097" grpc.time_ms=1.33 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1699vm-test-run-nixbot-gitlab> gitlab # [ 298.331882] gitaly[1368]: time="2026-07-01T08:10:24.118Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=13 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBN8GNCYNWC1S02XVVB39P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-01T08:10:34.032" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:24.102" grpc.time_ms=15.585 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1700vm-test-run-nixbot-gitlab> gitlab # [ 298.351844] gitaly[1368]: time="2026-07-01T08:10:24.171Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBN8GNCYNWC1S02XVVB39P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T08:10:34.100" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:24.171" grpc.time_ms=0.306 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1701vm-test-run-nixbot-gitlab> gitlab # [ 298.365858] gitaly[1368]: time="2026-07-01T08:10:24.184Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBN8GNCYNWC1S02XVVB39P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-01T08:10:34.113" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:24.184" grpc.time_ms=0.481 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1702vm-test-run-nixbot-gitlab> gitlab # [ 298.570890] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:24.439Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"dbe142dc99c663552ffda994","created_at":"2026-07-01T08:10:24.429Z","trace_propagation_headers":{"sentry-trace":"73117be61270440b93728abe508314ed-db44862417984ffc","baggage":"sentry-trace_id=73117be61270440b93728abe508314ed,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBN8GNCYNWC1S02XVVB39P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:24.436Z","job_size_bytes":5,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-dbe142dc99c663552ffda994: start","job_status":"start","queue_duration_s":0.002518,"scheduling_latency_s":0.002582}1703vm-test-run-nixbot-gitlab> gitlab # [ 298.672833] gitaly[1368]: time="2026-07-01T08:10:24.542Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBN8GNCYNWC1S02XVVB39P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-01T08:10:34.062" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:24.533" grpc.time_ms=9.231 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1704vm-test-run-nixbot-gitlab> gitlab # [ 298.796247] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:24.665Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"dbe142dc99c663552ffda994","created_at":"2026-07-01T08:10:24.429Z","trace_propagation_headers":{"sentry-trace":"73117be61270440b93728abe508314ed-db44862417984ffc","baggage":"sentry-trace_id=73117be61270440b93728abe508314ed,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBN8GNCYNWC1S02XVVB39P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:24.436Z","job_size_bytes":5,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-dbe142dc99c663552ffda994: done: 0.226759 sec","job_status":"done","queue_duration_s":0.002518,"scheduling_latency_s":0.002582,"redis_calls":12,"redis_duration_s":0.00912,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001185,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.007225,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00071,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.172152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00052098,"duration_s":0.226759,"completed_at":"2026-07-01T08:10:24.665Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031757,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1705vm-test-run-nixbot-gitlab> gitlab # [ 298.894691] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:24.667Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"fcdcc4d22488a4dcb8b53355","created_at":"2026-07-01T08:10:24.473Z","trace_propagation_headers":{"sentry-trace":"73117be61270440b93728abe508314ed-db44862417984ffc","baggage":"sentry-trace_id=73117be61270440b93728abe508314ed,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBN8GNCYNWC1S02XVVB39P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:24.479Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-fcdcc4d22488a4dcb8b53355: start","job_status":"start","queue_duration_s":0.18825,"scheduling_latency_s":0.188277}1706vm-test-run-nixbot-gitlab> gitlab # [ 299.098134] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:24.966Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"fcdcc4d22488a4dcb8b53355","created_at":"2026-07-01T08:10:24.473Z","trace_propagation_headers":{"sentry-trace":"73117be61270440b93728abe508314ed-db44862417984ffc","baggage":"sentry-trace_id=73117be61270440b93728abe508314ed,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBN8GNCYNWC1S02XVVB39P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:24.479Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-fcdcc4d22488a4dcb8b53355: done: 0.298408 sec","job_status":"done","queue_duration_s":0.18825,"scheduling_latency_s":0.188277,"redis_calls":11,"redis_duration_s":0.0035489999999999996,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000841,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002057,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000651,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.247,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.247,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.036,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.200041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00061038,"duration_s":0.298408,"completed_at":"2026-07-01T08:10:24.966Z","load_balancing_strategy":"primary","db_duration_s":0.104266,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1707vm-test-run-nixbot-gitlab> gitlab # [ 299.201092] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:24.969Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"e2368412bdb3cdd1f8488588","created_at":"2026-07-01T08:10:24.486Z","trace_propagation_headers":{"sentry-trace":"73117be61270440b93728abe508314ed-db44862417984ffc","baggage":"sentry-trace_id=73117be61270440b93728abe508314ed,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBN8GNCYNWC1S02XVVB39P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:24.501Z","job_size_bytes":89,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-e2368412bdb3cdd1f8488588: start","job_status":"start","queue_duration_s":0.468488,"scheduling_latency_s":0.468516}1708vm-test-run-nixbot-gitlab> gitlab # [ 299.225546] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:25.080Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"e2368412bdb3cdd1f8488588","created_at":"2026-07-01T08:10:24.486Z","trace_propagation_headers":{"sentry-trace":"73117be61270440b93728abe508314ed-db44862417984ffc","baggage":"sentry-trace_id=73117be61270440b93728abe508314ed,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBN8GNCYNWC1S02XVVB39P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:24.501Z","job_size_bytes":89,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-e2368412bdb3cdd1f8488588: done: 0.110188 sec","job_status":"done","queue_duration_s":0.468488,"scheduling_latency_s":0.468516,"redis_calls":11,"redis_duration_s":0.003305,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000722,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001894,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000689,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00057264,"duration_s":0.110188,"completed_at":"2026-07-01T08:10:25.080Z","load_balancing_strategy":"primary","db_duration_s":0.036036,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1709vm-test-run-nixbot-gitlab> gitlab # [ 299.286440] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:25 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 40301710vm-test-run-nixbot-gitlab> gitlab # [ 299.289074] setup-gitlab-start[3256]: + mkdir -p /tmp/test-flake1711vm-test-run-nixbot-gitlab> gitlab # [ 299.290100] setup-gitlab-start[3256]: + cd /tmp/test-flake1712vm-test-run-nixbot-gitlab> gitlab # [ 299.291020] setup-gitlab-start[3256]: + git init -b master1713vm-test-run-nixbot-gitlab> gitlab # [ 299.292099] setup-gitlab-start[3682]: Initialized empty Git repository in /tmp/test-flake/.git/1714vm-test-run-nixbot-gitlab> gitlab # [ 299.293514] setup-gitlab-start[3256]: + git config user.name test1715vm-test-run-nixbot-gitlab> gitlab # [ 299.294467] setup-gitlab-start[3256]: + git config user.email test@example.com1716vm-test-run-nixbot-gitlab> gitlab # [ 299.295663] setup-gitlab-start[3256]: + cat1717vm-test-run-nixbot-gitlab> gitlab # [ 299.299093] setup-gitlab-start[3256]: + git add flake.nix1718vm-test-run-nixbot-gitlab> gitlab # [ 299.306936] setup-gitlab-start[3256]: + git commit -m 'initial commit'1719vm-test-run-nixbot-gitlab> gitlab # [ 299.326430] setup-gitlab-start[3687]: [master (root-commit) 6d53dbc] initial commit1720vm-test-run-nixbot-gitlab> gitlab # [ 299.327995] setup-gitlab-start[3687]: 1 file changed, 10 insertions(+)1721vm-test-run-nixbot-gitlab> gitlab # [ 299.329374] setup-gitlab-start[3687]: create mode 100644 flake.nix1722vm-test-run-nixbot-gitlab> gitlab # [ 299.330652] setup-gitlab-start[3256]: + git remote add origin http://oauth2:glpat-cfCOHVx5pibUnm7_o_gmTm86MQp1OjEH.01.0w1hnds5f@gitlab/root/test-flake.git1723vm-test-run-nixbot-gitlab> gitlab # [ 299.338907] setup-gitlab-start[3256]: + git push -u origin master1724vm-test-run-nixbot-gitlab> gitlab # [ 299.652551] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:25 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 2631725vm-test-run-nixbot-gitlab> gitlab # [ 299.957677] gitaly[1368]: time="2026-07-01T08:10:25.826Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNCVKF39HHFPNP0THHHDQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:25.815" grpc.time_ms=10.859 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1726vm-test-run-nixbot-gitlab> gitlab # [ 299.998023] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:25 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 3111727vm-test-run-nixbot-gitlab> gitlab # [ 300.363781] gitaly[1368]: time="2026-07-01T08:10:26.233Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-01T08:10:36.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:26.217" grpc.time_ms=15.578 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1728vm-test-run-nixbot-gitlab> gitlab # [ 300.421201] gitaly[1368]: time="2026-07-01T08:10:26.291Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=3 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-01T08:10:56.071" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-01T08:10:26.272" grpc.time_ms=18.483 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1729vm-test-run-nixbot-gitlab> gitlab # [ 300.493140] gitaly[1368]: time="2026-07-01T08:10:26.363Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWEBND6MJWHMZKNXF9J89763 duration_ms=308ns method=POST pid=1368 status=200 url="http://unix/api/v4/internal/allowed"1730vm-test-run-nixbot-gitlab> gitlab # [ 300.525661] gitaly[1368]: time="2026-07-01T08:10:26.395Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWEBND6MJWHMZKNXF9J89763 duration_ms=31ns method=POST pid=1368 status=200 url="http://unix/api/v4/internal/pre_receive"1731vm-test-run-nixbot-gitlab> gitlab # [ 300.538633] gitaly[1368]: time="2026-07-01T08:10:26.396Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5317 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:26.053" grpc.time_ms=342.99 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1732vm-test-run-nixbot-gitlab> gitlab # [ 300.611196] gitaly[1368]: time="2026-07-01T08:10:26.481Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:26.479" grpc.time_ms=1.255 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1733vm-test-run-nixbot-gitlab> gitlab # [ 300.666744] gitaly[1368]: time="2026-07-01T08:10:26.536Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:26.535" grpc.time_ms=0.874 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1734vm-test-run-nixbot-gitlab> gitlab # [ 300.729077] gitaly[1368]: time="2026-07-01T08:10:26.598Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4788 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:26.597" grpc.time_ms=0.866 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1735vm-test-run-nixbot-gitlab> gitlab # [ 300.873908] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:26.742Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 6d53dbc4843491d6937d6e9373e2dd74d6ee8e92 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"ccbd7ff9d72aba8f9711f2db","created_at":"2026-07-01T08:10:26.701Z","trace_propagation_headers":{"sentry-trace":"aafbfc18479748388584be316429f248-245a0edc04ff46d0","baggage":"sentry-trace_id=aafbfc18479748388584be316429f248,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBND6MJWHMZKNXF9J89763","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:26.739Z","job_size_bytes":185,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-ccbd7ff9d72aba8f9711f2db: start","job_status":"start","queue_duration_s":0.002836,"scheduling_latency_s":0.002875}1736vm-test-run-nixbot-gitlab> gitlab # [ 300.923755] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:26.764Z","message":"changes: \"0000000000000000000000000000000000000000 6d53dbc4843491d6937d6e9373e2dd74d6ee8e92 refs/heads/master\\n\""}1737vm-test-run-nixbot-gitlab> gitlab # [ 300.929753] gitaly[1368]: time="2026-07-01T08:10:26.792Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-01T08:10:36.005" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:26.785" grpc.time_ms=7.03 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1738vm-test-run-nixbot-gitlab> gitlab # [ 300.954119] gitaly[1368]: time="2026-07-01T08:10:26.813Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=6 command.system_time_ms=2 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-01T08:10:36.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:26.801" grpc.time_ms=12.235 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1739vm-test-run-nixbot-gitlab> gitlab # [ 300.974815] gitaly[1368]: time="2026-07-01T08:10:26.816Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T08:10:36.028" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:26.808" grpc.time_ms=7.758 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1740vm-test-run-nixbot-gitlab> gitlab # [ 300.992516] gitaly[1368]: time="2026-07-01T08:10:26.848Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1435 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=11 command.system_time_ms=12 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-01T08:10:36.040" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:26.821" grpc.time_ms=27.125 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1741vm-test-run-nixbot-gitlab> gitlab # [ 301.010876] gitaly[1368]: time="2026-07-01T08:10:26.855Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1437 command.oublock=0 command.real_time_ms=26 command.spawn_token_fork_ms=9 command.system_time_ms=17 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-01T08:10:36.024" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:26.824" grpc.time_ms=30.579 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1742vm-test-run-nixbot-gitlab> gitlab # [ 301.029441] gitaly[1368]: time="2026-07-01T08:10:26.869Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWEBND6MJWHMZKNXF9J89763 duration_ms=217ns method=POST pid=1368 status=200 url="http://unix/api/v4/internal/post_receive"1743vm-test-run-nixbot-gitlab> gitlab # [ 301.033703] gitlab-workhorse[1300]: time="2026-07-01T08:10:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521744vm-test-run-nixbot-gitlab> gitlab # [ 301.036122] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:26 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 10031745vm-test-run-nixbot-gitlab> gitlab # [ 301.038791] systemd[1]: Finished setup-gitlab.service.1746vm-test-run-nixbot-gitlab> gitlab # [ 301.039873] setup-gitlab-start[3691]: To http://gitlab/root/test-flake.git1747vm-test-run-nixbot-gitlab> gitlab # [ 301.041094] setup-gitlab-start[3691]: * [new branch] master -> master1748vm-test-run-nixbot-gitlab> gitlab # [ 301.042226] setup-gitlab-start[3691]: branch 'master' set up to track 'origin/master'.1749vm-test-run-nixbot-gitlab> gitlab # [ 301.043475] gitaly[1368]: time="2026-07-01T08:10:26.870Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4811 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:26.651" grpc.time_ms=218.893 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1750vm-test-run-nixbot-gitlab> gitlab # [ 301.054988] gitaly[1368]: time="2026-07-01T08:10:26.879Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 diskcache=7a4f6c97-a027-442a-a277-2cc926f6b5a3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:25.958" pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1751vm-test-run-nixbot-gitlab> gitlab # [ 301.065774] systemd[1]: Starting nixbot CI service...1752vm-test-run-nixbot-gitlab> gitlab # [ 301.066607] gitaly[1368]: time="2026-07-01T08:10:26.879Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=311 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13496 command.oublock=32 command.real_time_ms=914 command.spawn_token_fork_ms=1 command.system_time_ms=172 command.user_time_ms=139 component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:25.958" grpc.time_ms=920.97 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1753vm-test-run-nixbot-gitlab> gitlab # [ 301.080707] gitaly[1368]: time="2026-07-01T08:10:26.893Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=6 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-01T08:10:36.103" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:26.883" grpc.time_ms=9.757 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1754vm-test-run-nixbot-gitlab> gitlab # [ 301.094854] gitaly[1368]: time="2026-07-01T08:10:26.929Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:26.929" grpc.time_ms=0.361 pid=1368 span.kind=server system=grpc1755vm-test-run-nixbot-gitlab> gitlab # [ 301.104787] gitaly[1368]: time="2026-07-01T08:10:26.955Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3939 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:26.955" grpc.time_ms=0.344 pid=1368 span.kind=server system=grpc1756vm-test-run-nixbot-gitlab> gitlab # [ 301.114584] gitaly[1368]: time="2026-07-01T08:10:26.964Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 diskcache=b04c5d1c-ac35-4584-82eb-517a3379ca61 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-01T08:10:36.121" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:26.902" pid=1368 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1757vm-test-run-nixbot-gitlab> gitlab # [ 301.125292] gitaly[1368]: time="2026-07-01T08:10:26.964Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=65 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4935 command.oublock=8 command.real_time_ms=58 command.spawn_token_fork_ms=0 command.system_time_ms=30 command.user_time_ms=35 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-01T08:10:36.121" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:26.902" grpc.time_ms=62.333 pid=1368 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1758vm-test-run-nixbot-gitlab> gitlab # [ 301.138890] gitaly[1368]: time="2026-07-01T08:10:26.976Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T08:10:36.196" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:26.976" grpc.time_ms=0.132 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1759vm-test-run-nixbot-gitlab> gitlab # [ 301.149786] gitaly[1368]: time="2026-07-01T08:10:26.994Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=861 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-01T08:10:36.200" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:26.980" grpc.time_ms=14.338 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1760vm-test-run-nixbot-gitlab> gitlab # [ 301.235120] gitaly[1368]: time="2026-07-01T08:10:27.105Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=313 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-07-01T08:10:57.096" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:27.096" grpc.time_ms=8.271 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1761vm-test-run-nixbot-gitlab> gitlab # [ 301.283052] gitaly[1368]: time="2026-07-01T08:10:27.153Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-01T08:10:37.002" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:27.152" grpc.time_ms=0.462 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1762vm-test-run-nixbot-gitlab> gitlab # [ 301.306701] gitaly[1368]: time="2026-07-01T08:10:27.163Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=315 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-01T08:10:37.005" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-07-01T08:10:27.156" grpc.time_ms=7.216 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1763vm-test-run-nixbot-gitlab> gitlab # [ 301.331442] gitaly[1368]: time="2026-07-01T08:10:27.197Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T08:10:37.037" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:27.187" grpc.time_ms=10.101 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1764vm-test-run-nixbot-gitlab> gitlab # [ 301.356424] gitaly[1368]: time="2026-07-01T08:10:27.213Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T08:10:37.053" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:27.203" grpc.time_ms=10.176 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1765vm-test-run-nixbot-gitlab> gitlab # [ 301.379362] gitaly[1368]: time="2026-07-01T08:10:27.243Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-01T08:10:57.229" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:27.229" grpc.time_ms=14.058 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1766vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit setup-gitlab.service, in 302.43 seconds)1767vm-test-run-nixbot-gitlab> gitlab # [ 301.914844] gitaly[1368]: time="2026-07-01T08:10:27.784Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-01T08:10:37.000" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:27.781" grpc.time_ms=3.069 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1768vm-test-run-nixbot-gitlab> gitlab: waiting for unit nixbot.service1769vm-test-run-nixbot-gitlab> gitlab # [ 301.989900] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:27.857Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 6d53dbc4843491d6937d6e9373e2dd74d6ee8e92 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"ccbd7ff9d72aba8f9711f2db","created_at":"2026-07-01T08:10:26.701Z","trace_propagation_headers":{"sentry-trace":"aafbfc18479748388584be316429f248-245a0edc04ff46d0","baggage":"sentry-trace_id=aafbfc18479748388584be316429f248,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBND6MJWHMZKNXF9J89763","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:26.739Z","job_size_bytes":185,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-ccbd7ff9d72aba8f9711f2db: done: 1.115521 sec","job_status":"done","queue_duration_s":0.002836,"scheduling_latency_s":0.002875,"gitaly_calls":14,"gitaly_duration_s":0.223833,"redis_calls":200,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.050635,"redis_read_bytes":938,"redis_write_bytes":19803,"redis_cache_calls":14,"redis_cache_duration_s":0.006163,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":32,"redis_feature_flag_duration_s":0.012505,"redis_feature_flag_read_bytes":540,"redis_feature_flag_write_bytes":4279,"redis_queues_calls":8,"redis_queues_duration_s":0.001132,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4936,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.005075,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":91,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.011668,"redis_repository_cache_read_bytes":230,"redis_repository_cache_write_bytes":6310,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.014092,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":51,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":51,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.067,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":51,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.368,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.402,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.067,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.763361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00052008,"exclusive_lock_hold_duration_s":0.0239767,"duration_s":1.115521,"completed_at":"2026-07-01T08:10:27.857Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.107798,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1770vm-test-run-nixbot-gitlab> gitlab # [ 302.045517] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:27.867Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"055b21d4ac94491d2450a0ec","created_at":"2026-07-01T08:10:26.850Z","trace_propagation_headers":{"sentry-trace":"aafbfc18479748388584be316429f248-fa65cb6f9b5647e0","baggage":"sentry-trace_id=aafbfc18479748388584be316429f248,sentry-release=Unknown"},"correlation_id":"01KWEBND6MJWHMZKNXF9J89763","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:26.857Z","job_size_bytes":46,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-055b21d4ac94491d2450a0ec: start","job_status":"start","queue_duration_s":1.010798,"scheduling_latency_s":1.010851}1771vm-test-run-nixbot-gitlab> gitlab # [ 302.101675] gitaly[1368]: time="2026-07-01T08:10:27.971Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-07-01T14:10:27.965" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:27.965" grpc.time_ms=4.688 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1772vm-test-run-nixbot-gitlab> gitlab # [ 302.126740] gitaly[1368]: time="2026-07-01T08:10:27.991Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=593 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-01T08:10:57.080" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:27.981" grpc.time_ms=10.302 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1773vm-test-run-nixbot-gitlab> gitlab # [ 302.159857] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:28.029Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"055b21d4ac94491d2450a0ec","created_at":"2026-07-01T08:10:26.850Z","trace_propagation_headers":{"sentry-trace":"aafbfc18479748388584be316429f248-fa65cb6f9b5647e0","baggage":"sentry-trace_id=aafbfc18479748388584be316429f248,sentry-release=Unknown"},"correlation_id":"01KWEBND6MJWHMZKNXF9J89763","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:26.857Z","job_size_bytes":46,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-055b21d4ac94491d2450a0ec: done: 0.161657 sec","job_status":"done","queue_duration_s":1.010798,"scheduling_latency_s":1.010851,"gitaly_calls":2,"gitaly_duration_s":0.020949,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.01371,"redis_read_bytes":68,"redis_write_bytes":4889,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.005008,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000878,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002431,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.00406,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":754,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001333,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.107797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00041752,"duration_s":0.161657,"completed_at":"2026-07-01T08:10:28.029Z","load_balancing_strategy":"primary","db_duration_s":0.018712,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1774vm-test-run-nixbot-gitlab> gitlab # [ 302.245848] nixbot[3802]: {"ts": "2026-07-01T08:10:28.051Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1775vm-test-run-nixbot-gitlab> gitlab # [ 302.249217] nixbot[3802]: {"ts": "2026-07-01T08:10:28.074Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1776vm-test-run-nixbot-gitlab> gitlab # [ 302.251923] nixbot[3802]: {"ts": "2026-07-01T08:10:28.093Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1777vm-test-run-nixbot-gitlab> gitlab # [ 302.254961] nixbot[3802]: {"ts": "2026-07-01T08:10:28.096Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1778vm-test-run-nixbot-gitlab> gitlab # [ 302.258125] nixbot[3802]: {"ts": "2026-07-01T08:10:28.103Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1779vm-test-run-nixbot-gitlab> gitlab # [ 302.260919] nixbot[3802]: {"ts": "2026-07-01T08:10:28.110Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1780vm-test-run-nixbot-gitlab> gitlab # [ 302.263731] nixbot[3802]: {"ts": "2026-07-01T08:10:28.113Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1781vm-test-run-nixbot-gitlab> gitlab # [ 302.266753] nixbot[3802]: {"ts": "2026-07-01T08:10:28.114Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1782vm-test-run-nixbot-gitlab> gitlab # [ 302.269579] gitaly[1368]: time="2026-07-01T08:10:28.094Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 diskcache=4b080c51-d4a0-4a5d-8de0-4875991572fc grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-01T14:10:28.061" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:28.062" pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1783vm-test-run-nixbot-gitlab> gitlab # [ 302.282606] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:28.031Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"6d35f408d1b51c4a0f5ce227","created_at":"2026-07-01T08:10:26.872Z","trace_propagation_headers":{"sentry-trace":"aafbfc18479748388584be316429f248-fa65cb6f9b5647e0","baggage":"sentry-trace_id=aafbfc18479748388584be316429f248,sentry-release=Unknown"},"correlation_id":"01KWEBND6MJWHMZKNXF9J89763","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:26.879Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-6d35f408d1b51c4a0f5ce227: start","job_status":"start","queue_duration_s":1.152093,"scheduling_latency_s":1.152121}1784vm-test-run-nixbot-gitlab> gitlab # [ 302.302131] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:28.136Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"6d35f408d1b51c4a0f5ce227","created_at":"2026-07-01T08:10:26.872Z","trace_propagation_headers":{"sentry-trace":"aafbfc18479748388584be316429f248-fa65cb6f9b5647e0","baggage":"sentry-trace_id=aafbfc18479748388584be316429f248,sentry-release=Unknown"},"correlation_id":"01KWEBND6MJWHMZKNXF9J89763","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:26.879Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-6d35f408d1b51c4a0f5ce227: done: 0.10489 sec","job_status":"done","queue_duration_s":1.152093,"scheduling_latency_s":1.152121,"gitaly_calls":1,"gitaly_duration_s":0.035285,"redis_calls":15,"redis_duration_s":0.00469,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00052,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001604,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001156,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00141,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.014,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.022,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.07549872,"duration_s":0.10489,"completed_at":"2026-07-01T08:10:28.136Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01876,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1785vm-test-run-nixbot-gitlab> gitlab # [ 302.353788] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:28.138Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"96636d769cbd6435244bfefc","created_at":"2026-07-01T08:10:26.972Z","trace_propagation_headers":{"sentry-trace":"aafbfc18479748388584be316429f248-fa65cb6f9b5647e0","baggage":"sentry-trace_id=aafbfc18479748388584be316429f248,sentry-release=Unknown"},"correlation_id":"01KWEBND6MJWHMZKNXF9J89763","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:26.974Z","job_size_bytes":23,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-96636d769cbd6435244bfefc: start","job_status":"start","queue_duration_s":1.16349,"scheduling_latency_s":1.163512}1786vm-test-run-nixbot-gitlab> gitlab # [ 302.370787] nixbot[3802]: {"ts": "2026-07-01T08:10:28.118Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1787vm-test-run-nixbot-gitlab> gitlab # [ 302.372979] nixbot[3802]: {"ts": "2026-07-01T08:10:28.119Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1788vm-test-run-nixbot-gitlab> gitlab # [ 302.375141] nixbot[3802]: {"ts": "2026-07-01T08:10:28.127Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1789vm-test-run-nixbot-gitlab> gitlab # [ 302.377288] nixbot[3802]: {"ts": "2026-07-01T08:10:28.131Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1790vm-test-run-nixbot-gitlab> gitlab # [ 302.379388] nixbot[3802]: {"ts": "2026-07-01T08:10:28.134Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1791vm-test-run-nixbot-gitlab> gitlab # [ 302.381659] nixbot[3802]: {"ts": "2026-07-01T08:10:28.136Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1792vm-test-run-nixbot-gitlab> gitlab # [ 302.383891] nixbot[3802]: {"ts": "2026-07-01T08:10:28.140Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1793vm-test-run-nixbot-gitlab> gitlab # [ 302.386208] nixbot[3802]: {"ts": "2026-07-01T08:10:28.142Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1794vm-test-run-nixbot-gitlab> gitlab # [ 302.388311] nixbot[3802]: {"ts": "2026-07-01T08:10:28.143Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1795vm-test-run-nixbot-gitlab> gitlab # [ 302.390419] nixbot[3802]: {"ts": "2026-07-01T08:10:28.145Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1796vm-test-run-nixbot-gitlab> gitlab # [ 302.392483] nixbot[3802]: {"ts": "2026-07-01T08:10:28.149Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1797vm-test-run-nixbot-gitlab> gitlab # [ 302.394554] nixbot[3802]: {"ts": "2026-07-01T08:10:28.151Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1798vm-test-run-nixbot-gitlab> gitlab # [ 302.396754] gitaly[1368]: time="2026-07-01T08:10:28.094Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1175 command.oublock=0 command.real_time_ms=36 command.spawn_token_fork_ms=4 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-01T14:10:28.061" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:28.062" grpc.time_ms=32.346 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1799vm-test-run-nixbot-gitlab> gitlab # [ 302.413333] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:28.203Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"96636d769cbd6435244bfefc","created_at":"2026-07-01T08:10:26.972Z","trace_propagation_headers":{"sentry-trace":"aafbfc18479748388584be316429f248-fa65cb6f9b5647e0","baggage":"sentry-trace_id=aafbfc18479748388584be316429f248,sentry-release=Unknown"},"correlation_id":"01KWEBND6MJWHMZKNXF9J89763","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:26.974Z","job_size_bytes":23,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-96636d769cbd6435244bfefc: done: 0.065186 sec","job_status":"done","queue_duration_s":1.16349,"scheduling_latency_s":1.163512,"gitaly_calls":1,"gitaly_duration_s":0.027717,"redis_calls":16,"redis_duration_s":0.004804,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000388,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001282,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001738,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001396,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.065186,"completed_at":"2026-07-01T08:10:28.203Z","load_balancing_strategy":"primary","db_duration_s":0.002382,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1800vm-test-run-nixbot-gitlab> gitlab # [ 302.459128] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:28.204Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"8c36ce961741086d76e47719","created_at":"2026-07-01T08:10:27.692Z","trace_propagation_headers":{"sentry-trace":"aafbfc18479748388584be316429f248-fa65cb6f9b5647e0","baggage":"sentry-trace_id=aafbfc18479748388584be316429f248,sentry-release=Unknown"},"correlation_id":"01KWEBND6MJWHMZKNXF9J89763","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd9fff3d548b275a8481b8115cda7e66879b0fce72bf8f146854777eda112dda","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:27.704Z","job_size_bytes":467,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-8c36ce961741086d76e47719: start","job_status":"start","queue_duration_s":0.500499,"scheduling_latency_s":0.500521}1801vm-test-run-nixbot-gitlab> gitlab # [ 302.476741] gitaly[1368]: time="2026-07-01T08:10:28.194Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=593 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-01T08:10:58.167" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:28.168" grpc.time_ms=25.586 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1802vm-test-run-nixbot-gitlab> gitlab # [ 302.490151] gitaly[1368]: time="2026-07-01T08:10:28.275Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:10:58.274" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:28.274" grpc.time_ms=0.854 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1803vm-test-run-nixbot-gitlab> gitlab # [ 302.502990] gitaly[1368]: time="2026-07-01T08:10:28.294Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-01T08:10:38.128" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:28.278" grpc.time_ms=15.447 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1804vm-test-run-nixbot-gitlab> gitlab # [ 302.586269] gitaly[1368]: time="2026-07-01T08:10:28.456Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-01T08:10:58.454" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:28.454" grpc.time_ms=1.39 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1805vm-test-run-nixbot-gitlab> gitlab # [ 302.602560] gitaly[1368]: time="2026-07-01T08:10:28.471Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=47 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-01T08:10:58.464" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:28.465" grpc.time_ms=5.74 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1806vm-test-run-nixbot-gitlab> gitlab # [ 302.738279] nixbot[3802]: INFO: Started server process [3802]1807vm-test-run-nixbot-gitlab> gitlab # [ 302.741083] nixbot[3802]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1808vm-test-run-nixbot-gitlab> gitlab # [ 302.750251] nixbot[3802]: INFO: ::ffff:127.0.0.1:45104 - "GET /health HTTP/1.1" 200 OK1809vm-test-run-nixbot-gitlab> gitlab # [ 302.758731] systemd[1]: Started nixbot CI service.1810vm-test-run-nixbot-gitlab> gitlab # [ 302.763760] systemd[1]: Reached target Multi-User System.1811vm-test-run-nixbot-gitlab> gitlab # [ 302.766521] systemd[1]: Startup finished in 778ms (kernel) + 5.005s (initrd) + 4min 56.976s (userspace) = 5min 2.760s.1812vm-test-run-nixbot-gitlab> gitlab # [ 302.786397] gitaly[1368]: time="2026-07-01T08:10:28.656Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T08:10:38.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:28.645" grpc.time_ms=11.098 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1813vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit nixbot.service, in 1.09 seconds)1814vm-test-run-nixbot-gitlab> gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1815vm-test-run-nixbot-gitlab> gitlab # [ 303.114128] nixbot[3802]: INFO: ::ffff:127.0.0.1:45112 - "GET /health HTTP/1.1" 200 OK1816vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1817vm-test-run-nixbot-gitlab> subtest: project discovered and webhook registered1818vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1819vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.19 seconds)1820vm-test-run-nixbot-gitlab> []1821vm-test-run-nixbot-gitlab> gitlab # [ 303.305897] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:29 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 1421822vm-test-run-nixbot-gitlab> gitlab # [ 303.551588] gitaly[1368]: time="2026-07-01T08:10:29.420Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=3 command.system_time_ms=13 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNFW9N0V0YV9YWW6XGA3Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-01T08:10:59.191" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:29.392" grpc.time_ms=28.426 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1823vm-test-run-nixbot-gitlab> gitlab # [ 303.643437] gitaly[1368]: time="2026-07-01T08:10:29.513Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-01T08:10:59.491" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:29.491" grpc.time_ms=21.434 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1824vm-test-run-nixbot-gitlab> gitlab # [ 303.714334] gitaly[1368]: time="2026-07-01T08:10:29.584Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBND6MJWHMZKNXF9J89763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-01T08:10:59.558" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:29.559" grpc.time_ms=24.429 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1825vm-test-run-nixbot-gitlab> gitlab # [ 303.803365] sidekiq-cluster[2429]: {"severity":"WARN","time":"2026-07-01T08:10:29.672Z","class":"CreatePipelineWorker","correlation_id":"01KWEBND6MJWHMZKNXF9J89763","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"6d53dbc4843491d6937d6e9373e2dd74d6ee8e92","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"6d53dbc4843491d6937d6e9373e2dd74d6ee8e92","ref":"refs/heads/master"},"retry":0}1826vm-test-run-nixbot-gitlab> gitlab # [ 303.823135] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:29.691Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"8c36ce961741086d76e47719","created_at":"2026-07-01T08:10:27.692Z","trace_propagation_headers":{"sentry-trace":"aafbfc18479748388584be316429f248-fa65cb6f9b5647e0","baggage":"sentry-trace_id=aafbfc18479748388584be316429f248,sentry-release=Unknown"},"correlation_id":"01KWEBND6MJWHMZKNXF9J89763","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd9fff3d548b275a8481b8115cda7e66879b0fce72bf8f146854777eda112dda","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:27.704Z","job_size_bytes":467,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-8c36ce961741086d76e47719: done: 1.486338 sec","job_status":"done","queue_duration_s":0.500499,"scheduling_latency_s":0.500521,"gitaly_calls":7,"gitaly_duration_s":0.146941,"redis_calls":53,"redis_duration_s":0.015481,"redis_read_bytes":171,"redis_write_bytes":4426,"redis_cache_calls":4,"redis_cache_duration_s":0.001311,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.006537,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002354,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003574,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001705,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":40,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":40,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.063,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.05,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.232226,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0010198,"duration_s":1.486338,"completed_at":"2026-07-01T08:10:29.691Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.111831,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1827vm-test-run-nixbot-gitlab> gitlab # [ 303.907447] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:29.694Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"9190b8d5b72c282aa29c677f","created_at":"2026-07-01T08:10:28.004Z","trace_propagation_headers":{"sentry-trace":"aafbfc18479748388584be316429f248-107d90025e7b4aef","baggage":"sentry-trace_id=aafbfc18479748388584be316429f248,sentry-release=Unknown"},"correlation_id":"01KWEBND6MJWHMZKNXF9J89763","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:28.012Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-9190b8d5b72c282aa29c677f: start","job_status":"start","queue_duration_s":1.682231,"scheduling_latency_s":1.682277}1828vm-test-run-nixbot-gitlab> gitlab # [ 303.928084] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:29.791Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"9190b8d5b72c282aa29c677f","created_at":"2026-07-01T08:10:28.004Z","trace_propagation_headers":{"sentry-trace":"aafbfc18479748388584be316429f248-107d90025e7b4aef","baggage":"sentry-trace_id=aafbfc18479748388584be316429f248,sentry-release=Unknown"},"correlation_id":"01KWEBND6MJWHMZKNXF9J89763","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:28.012Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-9190b8d5b72c282aa29c677f: done: 0.09725 sec","job_status":"done","queue_duration_s":1.682231,"scheduling_latency_s":1.682277,"redis_calls":19,"redis_duration_s":0.00791,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001361,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.001372,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003892,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001285,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067422,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01833846,"duration_s":0.09725,"completed_at":"2026-07-01T08:10:29.791Z","load_balancing_strategy":"primary","db_duration_s":0.014026,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1829vm-test-run-nixbot-gitlab> gitlab # [ 303.975566] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:29.794Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"c834e6fb32e552d8bc880856","created_at":"2026-07-01T08:10:29.761Z","trace_propagation_headers":{"sentry-trace":"aafbfc18479748388584be316429f248-02fd4945682347ac","baggage":"sentry-trace_id=aafbfc18479748388584be316429f248,sentry-release=Unknown"},"correlation_id":"01KWEBND6MJWHMZKNXF9J89763","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:29.773Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-c834e6fb32e552d8bc880856: start","job_status":"start","queue_duration_s":0.020593,"scheduling_latency_s":0.020622}1830vm-test-run-nixbot-gitlab> gitlab # [ 303.993191] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:29 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 11971831vm-test-run-nixbot-gitlab> gitlab # [ 303.995777] nixbot[3802]: {"ts": "2026-07-01T08:10:29.819Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1832vm-test-run-nixbot-gitlab> gitlab # [ 303.998363] nixbot[3802]: {"ts": "2026-07-01T08:10:29.842Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1833vm-test-run-nixbot-gitlab> gitlab # [ 304.041951] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:29 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 591834vm-test-run-nixbot-gitlab> gitlab # [ 304.044686] nixbot[3802]: {"ts": "2026-07-01T08:10:29.913Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1835vm-test-run-nixbot-gitlab> gitlab # [ 304.047252] nixbot[3802]: {"ts": "2026-07-01T08:10:29.915Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1836vm-test-run-nixbot-gitlab> gitlab # [ 304.066289] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:29.935Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"c834e6fb32e552d8bc880856","created_at":"2026-07-01T08:10:29.761Z","trace_propagation_headers":{"sentry-trace":"aafbfc18479748388584be316429f248-02fd4945682347ac","baggage":"sentry-trace_id=aafbfc18479748388584be316429f248,sentry-release=Unknown"},"correlation_id":"01KWEBND6MJWHMZKNXF9J89763","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:29.773Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-c834e6fb32e552d8bc880856: done: 0.141401 sec","job_status":"done","queue_duration_s":0.020593,"scheduling_latency_s":0.020622,"redis_calls":11,"redis_duration_s":0.0032559999999999998,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000552,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00209,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000614,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.11163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000461,"duration_s":0.141401,"completed_at":"2026-07-01T08:10:29.935Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024025,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1837vm-test-run-nixbot-gitlab> gitlab # [ 304.253878] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 2051838vm-test-run-nixbot-gitlab> gitlab # [ 304.260658] nixbot[3802]: {"ts": "2026-07-01T08:10:30.125Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1839vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1840vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.16 seconds)1841vm-test-run-nixbot-gitlab> [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-07-01T08:10:30.106Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]1842vm-test-run-nixbot-gitlab> (finished: subtest: project discovered and webhook registered, in 1.35 seconds)1843vm-test-run-nixbot-gitlab> subtest: push delivers the webhook1844vm-test-run-nixbot-gitlab> gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1845vm-test-run-nixbot-gitlab> gitlab # [ 304.472469] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 991846vm-test-run-nixbot-gitlab> gitlab # [ 304.576289] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:30 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 3061847vm-test-run-nixbot-gitlab> gitlab # [ 304.582455] nixbot[3802]: {"ts": "2026-07-01T08:10:30.446Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1848vm-test-run-nixbot-gitlab> gitlab # [ 304.633306] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:30 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 771849vm-test-run-nixbot-gitlab> gitlab # [ 304.749498] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 1421850vm-test-run-nixbot-gitlab> gitlab # [ 304.754788] nixbot[3802]: {"ts": "2026-07-01T08:10:30.622Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1851vm-test-run-nixbot-gitlab> gitlab # [ 304.757834] gitaly[1368]: time="2026-07-01T08:10:30.625Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHQ6C645MS8GZKDN2KKE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:30.614" grpc.time_ms=10.795 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1852vm-test-run-nixbot-gitlab> gitlab # Enumerating objects: 5, done.1853vm-test-run-nixbot-gitlab> gitlab # [ 304.771851] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:30 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 203 "" "git/2.54.0" 1231854vm-test-run-nixbot-gitlab> gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1855vm-test-run-nixbot-gitlab> gitlab # Delta compression using up to 4 threads1856vm-test-run-nixbot-gitlab> gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1857vm-test-run-nixbot-gitlab> gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 136.00 KiB/s, done.1858vm-test-run-nixbot-gitlab> gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1859vm-test-run-nixbot-gitlab> gitlab # [ 304.833720] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:30 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 761860vm-test-run-nixbot-gitlab> gitlab # [ 304.841977] nixbot[3802]: {"ts": "2026-07-01T08:10:30.704Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1861vm-test-run-nixbot-gitlab> gitlab # [ 304.931855] gitaly[1368]: time="2026-07-01T08:10:30.802Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNHXP3ZMT7S168WZKFNE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-07-01T08:11:00.086" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:30.787" grpc.time_ms=13.701 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1862vm-test-run-nixbot-gitlab> gitlab # [ 304.963866] gitaly[1368]: time="2026-07-01T08:10:30.813Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNHXP3ZMT7S168WZKFNE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:00.112" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:30.813" grpc.time_ms=0.481 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1863vm-test-run-nixbot-gitlab> gitlab # [ 304.985669] gitaly[1368]: time="2026-07-01T08:10:30.848Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=5 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNHXP3ZMT7S168WZKFNE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-01T08:10:40.007" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:30.837" grpc.time_ms=10.411 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1864vm-test-run-nixbot-gitlab> gitlab # [ 305.002670] gitaly[1368]: time="2026-07-01T08:10:30.862Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=592 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=12 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-01T08:10:40.046" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:30.846" grpc.time_ms=14.866 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1865vm-test-run-nixbot-gitlab> gitlab # [ 305.033408] gitaly[1368]: time="2026-07-01T08:10:30.903Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=5 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-01T08:11:00.695" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-01T08:10:30.895" grpc.time_ms=7.96 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1866vm-test-run-nixbot-gitlab> gitlab # [ 305.054321] gitaly[1368]: time="2026-07-01T08:10:30.916Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX duration_ms=120ns method=POST pid=1368 status=200 url="http://unix/api/v4/internal/allowed"1867vm-test-run-nixbot-gitlab> gitlab # [ 305.058599] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 2091868vm-test-run-nixbot-gitlab> gitlab # [ 305.062395] nixbot[3802]: {"ts": "2026-07-01T08:10:30.917Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1869vm-test-run-nixbot-gitlab> gitlab # [ 305.070925] gitaly[1368]: time="2026-07-01T08:10:30.941Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX duration_ms=24ns method=POST pid=1368 status=200 url="http://unix/api/v4/internal/pre_receive"1870vm-test-run-nixbot-gitlab> gitlab # [ 305.074926] gitaly[1368]: time="2026-07-01T08:10:30.941Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5317 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:30.795" grpc.time_ms=146.224 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1871vm-test-run-nixbot-gitlab> gitlab # [ 305.097405] gitaly[1368]: time="2026-07-01T08:10:30.967Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:30.966" grpc.time_ms=0.84 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1872vm-test-run-nixbot-gitlab> gitlab # [ 305.121401] gitaly[1368]: time="2026-07-01T08:10:30.991Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:30.991" grpc.time_ms=0.425 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1873vm-test-run-nixbot-gitlab> gitlab # [ 305.143830] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 911874vm-test-run-nixbot-gitlab> gitlab # [ 305.146608] nixbot[3802]: {"ts": "2026-07-01T08:10:31.013Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1875vm-test-run-nixbot-gitlab> gitlab # [ 305.149451] nixbot[3802]: {"ts": "2026-07-01T08:10:31.016Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1876vm-test-run-nixbot-gitlab> gitlab # [ 305.152333] gitaly[1368]: time="2026-07-01T08:10:31.022Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4788 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:31.021" grpc.time_ms=0.671 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1877vm-test-run-nixbot-gitlab> gitlab # [ 305.216464] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:31.083Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","6d53dbc4843491d6937d6e9373e2dd74d6ee8e92 70946e52cb9c0a8a279a04d1c7834a4d85e18e8b refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"14163e16e6ff6681147d776a","created_at":"2026-07-01T08:10:31.078Z","trace_propagation_headers":{"sentry-trace":"e596f5905f534fbaaa19d0fb1c1e5a9f-610ef7984d584e94","baggage":"sentry-trace_id=e596f5905f534fbaaa19d0fb1c1e5a9f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNHW2CQ14MB40HPAYR9MX","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:31.081Z","job_size_bytes":185,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-14163e16e6ff6681147d776a: start","job_status":"start","queue_duration_s":0.00153,"scheduling_latency_s":0.00157}1878vm-test-run-nixbot-gitlab> gitlab # [ 305.248301] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:31.110Z","message":"changes: \"6d53dbc4843491d6937d6e9373e2dd74d6ee8e92 70946e52cb9c0a8a279a04d1c7834a4d85e18e8b refs/heads/master\\n\""}1879vm-test-run-nixbot-gitlab> gitlab # [ 305.256379] gitaly[1368]: time="2026-07-01T08:10:31.126Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX duration_ms=78ns method=POST pid=1368 status=200 url="http://unix/api/v4/internal/post_receive"1880vm-test-run-nixbot-gitlab> gitlab # [ 305.259853] gitaly[1368]: time="2026-07-01T08:10:31.130Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4811 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:31.047" grpc.time_ms=82.487 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1881vm-test-run-nixbot-gitlab> gitlab # [ 305.279640] gitaly[1368]: time="2026-07-01T08:10:31.149Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=34 command.spawn_token_fork_ms=7 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNJ8BANY0ZYQ12YQ0FH0E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:31.113" grpc.time_ms=36.716 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc1882vm-test-run-nixbot-gitlab> gitlab # [ 305.293683] gitaly[1368]: time="2026-07-01T08:10:31.151Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=4 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T08:10:41.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:31.141" grpc.time_ms=10.3 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1883vm-test-run-nixbot-gitlab> gitlab # [ 305.307643] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:31 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 981884vm-test-run-nixbot-gitlab> gitlab # [ 305.310114] gitaly[1368]: time="2026-07-01T08:10:31.169Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=5 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-01T08:10:41.026" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:31.156" grpc.time_ms=13.246 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1885vm-test-run-nixbot-gitlab> gitlab # [ 305.324831] gitaly[1368]: time="2026-07-01T08:10:31.195Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX diskcache=d764ce9e-68d0-42a5-b78e-174c25eb4844 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:30.741" pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1886vm-test-run-nixbot-gitlab> gitlab # [ 305.335192] gitaly[1368]: time="2026-07-01T08:10:31.195Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=142 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=13066 command.oublock=32 command.real_time_ms=408 command.spawn_token_fork_ms=1 command.system_time_ms=79 command.user_time_ms=63 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:30.741" grpc.time_ms=454.012 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1887vm-test-run-nixbot-gitlab> gitlab # To http://gitlab/root/test-flake.git1888vm-test-run-nixbot-gitlab> gitlab # [ 305.351507] gitlab-workhorse[1300]: time="2026-07-01T08:10:31Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521889vm-test-run-nixbot-gitlab> gitlab # 6d53dbc..70946e5 [ 305.353928] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:31 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 5391890vm-test-run-nixbot-gitlab> gitlab # master -> master1891vm-test-run-nixbot-gitlab> gitlab # [ 305.356023] gitaly[1368]: time="2026-07-01T08:10:31.205Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-01T08:10:41.063" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:31.193" grpc.time_ms=11.957 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1892vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.91 seconds)1893vm-test-run-nixbot-gitlab> gitlab: must succeed: git -C /tmp/test-flake rev-parse master1894vm-test-run-nixbot-gitlab> gitlab # [ 305.371310] gitaly[1368]: time="2026-07-01T08:10:31.241Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=599 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=9 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-07-01T08:11:01.010" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:31.210" grpc.time_ms=18.873 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1895vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds)1896vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=70946e52cb9c0a8a279a04d1c7834a4d85e18e8b'1897vm-test-run-nixbot-gitlab> gitlab # [ 305.389388] gitaly[1368]: time="2026-07-01T08:10:31.246Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=318 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-01T08:10:41.110" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-07-01T08:10:31.240" grpc.time_ms=5.717 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1898vm-test-run-nixbot-gitlab> gitlab # [ 305.404066] gitaly[1368]: time="2026-07-01T08:10:31.262Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=7 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T08:10:41.123" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:31.253" grpc.time_ms=9.065 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1899vm-test-run-nixbot-gitlab> gitlab # [ 305.419576] gitaly[1368]: time="2026-07-01T08:10:31.277Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNJBJET636N6ZTQSKFSA6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:31.267" pid=1368 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc1900vm-test-run-nixbot-gitlab> gitlab # [ 305.430587] gitlab-workhorse[1300]: time="2026-07-01T08:10:31Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1491901vm-test-run-nixbot-gitlab> gitlab # [ 305.432714] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:31 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 1231902vm-test-run-nixbot-gitlab> gitlab # [ 305.434670] gitaly[1368]: time="2026-07-01T08:10:31.277Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNJBJET636N6ZTQSKFSA6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:31.267" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc1903vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=70946e52cb9c0a8a279a04d1c7834a4d85e18e8b', in 0.06 seconds)1904vm-test-run-nixbot-gitlab> []1905vm-test-run-nixbot-gitlab> gitlab # [ 305.449072] gitaly[1368]: time="2026-07-01T08:10:31.277Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNJBJET636N6ZTQSKFSA6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:31.267" grpc.time_ms=9.543 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc1906vm-test-run-nixbot-gitlab> gitlab # [ 305.463918] nixbot[3802]: INFO: ::ffff:127.0.0.1:45122 - "GET /api/repos/gitlab/root/test-flake/builds?commit=70946e52cb9c0a8a279a04d1c7834a4d85e18e8b HTTP/1.1" 200 OK1907vm-test-run-nixbot-gitlab> gitlab # [ 305.485788] gitaly[1368]: time="2026-07-01T08:10:31.356Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNJFG8JD31Y5YJQZ13SFA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:31.341" pid=1368 remote_ip="2001:db8:1::1" request_sha=a5f2250cd6ce41e3d1c4386dbb76f5cc24b54c02 response_bytes=591 span.kind=server system=grpc1908vm-test-run-nixbot-gitlab> gitlab # [ 305.497389] gitaly[1368]: time="2026-07-01T08:10:31.356Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1139 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNJFG8JD31Y5YJQZ13SFA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:31.341" negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc1909vm-test-run-nixbot-gitlab> gitlab # [ 305.511619] gitlab-workhorse[1300]: time="2026-07-01T08:10:31Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=5911910vm-test-run-nixbot-gitlab> gitlab # [ 305.513680] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:31 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 591 "" "git/2.54.0" 751911vm-test-run-nixbot-gitlab> gitlab # [ 305.515611] gitaly[1368]: time="2026-07-01T08:10:31.356Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1139 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNJFG8JD31Y5YJQZ13SFA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=9 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:31.341" grpc.time_ms=14.543 negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc1912vm-test-run-nixbot-gitlab> gitlab # [ 305.585219] gitaly[1368]: time="2026-07-01T08:10:31.455Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=9 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNJJZFWQ1JQFAB9RN4AEV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:31.438" grpc.time_ms=16.308 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc1913vm-test-run-nixbot-gitlab> gitlab # [ 305.601940] gitaly[1368]: time="2026-07-01T08:10:31.458Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-01T08:10:41.007" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:31.457" grpc.time_ms=1.178 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1914vm-test-run-nixbot-gitlab> gitlab # [ 305.617217] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:31 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 641915vm-test-run-nixbot-gitlab> gitlab # [ 305.619547] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:31.479Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","6d53dbc4843491d6937d6e9373e2dd74d6ee8e92 70946e52cb9c0a8a279a04d1c7834a4d85e18e8b refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"14163e16e6ff6681147d776a","created_at":"2026-07-01T08:10:31.078Z","trace_propagation_headers":{"sentry-trace":"e596f5905f534fbaaa19d0fb1c1e5a9f-610ef7984d584e94","baggage":"sentry-trace_id=e596f5905f534fbaaa19d0fb1c1e5a9f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNHW2CQ14MB40HPAYR9MX","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-01T08:10:31.081Z","job_size_bytes":185,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-14163e16e6ff6681147d776a: done: 0.396496 sec","job_status":"done","queue_duration_s":0.00153,"scheduling_latency_s":0.00157,"gitaly_calls":7,"gitaly_duration_s":0.093107,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.027460000000000002,"redis_read_bytes":350,"redis_write_bytes":13065,"redis_cache_calls":9,"redis_cache_duration_s":0.001686,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000534,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.00242,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6765,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001836,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.00915,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.011834,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.02,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.02,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.035,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.239254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00084772,"exclusive_lock_hold_duration_s":0.02622326,"duration_s":0.396496,"completed_at":"2026-07-01T08:10:31.479Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.040836,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1916vm-test-run-nixbot-gitlab> gitlab # [ 305.672605] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:31.481Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"8e09c9cc9a4d650da76a94e6","created_at":"2026-07-01T08:10:31.175Z","trace_propagation_headers":{"sentry-trace":"e596f5905f534fbaaa19d0fb1c1e5a9f-a77b3d665a174671","baggage":"sentry-trace_id=e596f5905f534fbaaa19d0fb1c1e5a9f,sentry-release=Unknown"},"correlation_id":"01KWEBNHW2CQ14MB40HPAYR9MX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:31.177Z","job_size_bytes":46,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8e09c9cc9a4d650da76a94e6: start","job_status":"start","queue_duration_s":0.303981,"scheduling_latency_s":0.304006}1917vm-test-run-nixbot-gitlab> gitlab # [ 305.689863] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:31.506Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"8e09c9cc9a4d650da76a94e6","created_at":"2026-07-01T08:10:31.175Z","trace_propagation_headers":{"sentry-trace":"e596f5905f534fbaaa19d0fb1c1e5a9f-a77b3d665a174671","baggage":"sentry-trace_id=e596f5905f534fbaaa19d0fb1c1e5a9f,sentry-release=Unknown"},"correlation_id":"01KWEBNHW2CQ14MB40HPAYR9MX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:31.177Z","job_size_bytes":46,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8e09c9cc9a4d650da76a94e6: done: 0.025353 sec","job_status":"done","queue_duration_s":0.303981,"scheduling_latency_s":0.304006,"redis_calls":9,"redis_duration_s":0.004528,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003009,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000357,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001162,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025353,"completed_at":"2026-07-01T08:10:31.506Z","load_balancing_strategy":"primary","db_duration_s":0.002265,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1918vm-test-run-nixbot-gitlab> gitlab # [ 305.735309] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:31.508Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b6ec30eb294f4ced11303680","created_at":"2026-07-01T08:10:31.186Z","trace_propagation_headers":{"sentry-trace":"e596f5905f534fbaaa19d0fb1c1e5a9f-a77b3d665a174671","baggage":"sentry-trace_id=e596f5905f534fbaaa19d0fb1c1e5a9f,sentry-release=Unknown"},"correlation_id":"01KWEBNHW2CQ14MB40HPAYR9MX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:31.188Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b6ec30eb294f4ced11303680: start","job_status":"start","queue_duration_s":0.319306,"scheduling_latency_s":0.31933}1919vm-test-run-nixbot-gitlab> gitlab # [ 305.752177] gitlab-workhorse[1300]: time="2026-07-01T08:10:31Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671920vm-test-run-nixbot-gitlab> gitlab # [ 305.753985] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:31 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 661921vm-test-run-nixbot-gitlab> gitlab # [ 305.755837] gitaly[1368]: time="2026-07-01T08:10:31.525Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNJN3K4BCRV3M0Z82WK0V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:31.516" pid=1368 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1922vm-test-run-nixbot-gitlab> gitlab # [ 305.766282] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:31.569Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b6ec30eb294f4ced11303680","created_at":"2026-07-01T08:10:31.186Z","trace_propagation_headers":{"sentry-trace":"e596f5905f534fbaaa19d0fb1c1e5a9f-a77b3d665a174671","baggage":"sentry-trace_id=e596f5905f534fbaaa19d0fb1c1e5a9f,sentry-release=Unknown"},"correlation_id":"01KWEBNHW2CQ14MB40HPAYR9MX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:31.188Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b6ec30eb294f4ced11303680: done: 0.061668 sec","job_status":"done","queue_duration_s":0.319306,"scheduling_latency_s":0.31933,"gitaly_calls":1,"gitaly_duration_s":0.030658,"redis_calls":12,"redis_duration_s":0.001823,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000752,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000341,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00073,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.04331622,"duration_s":0.061668,"completed_at":"2026-07-01T08:10:31.569Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003767,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1923vm-test-run-nixbot-gitlab> gitlab # [ 305.812083] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:31.571Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"88389dee2e583a4b5532a58f","created_at":"2026-07-01T08:10:31.310Z","trace_propagation_headers":{"sentry-trace":"e596f5905f534fbaaa19d0fb1c1e5a9f-a77b3d665a174671","baggage":"sentry-trace_id=e596f5905f534fbaaa19d0fb1c1e5a9f,sentry-release=Unknown"},"correlation_id":"01KWEBNHW2CQ14MB40HPAYR9MX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9c1a468fb18824b60346d06799f8b73e42feeffcabc689977a6830df7f6168d8","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:31.314Z","job_size_bytes":467,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-88389dee2e583a4b5532a58f: start","job_status":"start","queue_duration_s":0.256029,"scheduling_latency_s":0.256048}1924vm-test-run-nixbot-gitlab> gitlab # [ 305.829995] gitaly[1368]: time="2026-07-01T08:10:31.525Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNJN3K4BCRV3M0Z82WK0V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:31.516" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc1925vm-test-run-nixbot-gitlab> gitlab # [ 305.843870] gitaly[1368]: time="2026-07-01T08:10:31.525Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNJN3K4BCRV3M0Z82WK0V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:31.516" grpc.time_ms=9.367 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc1926vm-test-run-nixbot-gitlab> gitlab # [ 305.858636] gitaly[1368]: time="2026-07-01T08:10:31.554Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX diskcache=9b2df301-3931-4029-b345-24c5949c0c4a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-01T14:10:31.526" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:31.526" pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1927vm-test-run-nixbot-gitlab> gitlab # [ 305.868928] gitaly[1368]: time="2026-07-01T08:10:31.555Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1512 command.oublock=0 command.real_time_ms=32 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-01T14:10:31.526" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:31.526" grpc.time_ms=28.319 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1928vm-test-run-nixbot-gitlab> gitlab # [ 305.884486] gitaly[1368]: time="2026-07-01T08:10:31.594Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:01.392" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:31.592" grpc.time_ms=1.403 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1929vm-test-run-nixbot-gitlab> gitlab # [ 305.898449] gitaly[1368]: time="2026-07-01T08:10:31.604Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-01T08:10:41.146" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:31.596" grpc.time_ms=7.377 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1930vm-test-run-nixbot-gitlab> gitlab # [ 305.911566] gitaly[1368]: time="2026-07-01T08:10:31.613Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNJR9DB27KQNSE26RWSFY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:01.412" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:31.612" grpc.time_ms=0.652 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1931vm-test-run-nixbot-gitlab> gitlab # [ 305.924767] gitaly[1368]: time="2026-07-01T08:10:31.640Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-01T08:11:01.439" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:31.639" grpc.time_ms=0.812 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1932vm-test-run-nixbot-gitlab> gitlab # [ 305.938264] gitaly[1368]: time="2026-07-01T08:10:31.645Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=48 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-01T08:11:01.443" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:31.643" grpc.time_ms=2.252 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1933vm-test-run-nixbot-gitlab> gitlab # [ 305.954708] gitaly[1368]: time="2026-07-01T08:10:31.657Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=592 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNJR9DB27KQNSE26RWSFY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-01T08:11:01.449" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:31.649" grpc.time_ms=8.107 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1934vm-test-run-nixbot-gitlab> gitlab # [ 305.968865] gitaly[1368]: time="2026-07-01T08:10:31.673Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNJR9DB27KQNSE26RWSFY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T08:10:41.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:31.665" grpc.time_ms=7.745 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1935vm-test-run-nixbot-gitlab> gitlab # [ 305.982546] gitaly[1368]: time="2026-07-01T08:10:31.694Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T08:10:41.047" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:31.687" grpc.time_ms=6.776 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1936vm-test-run-nixbot-gitlab> gitlab # [ 305.996099] gitaly[1368]: time="2026-07-01T08:10:31.735Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNJR9DB27KQNSE26RWSFY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-01T08:10:41.074" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:31.734" grpc.time_ms=0.265 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1937vm-test-run-nixbot-gitlab> gitlab # [ 306.006967] gitaly[1368]: time="2026-07-01T08:10:31.748Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNJR9DB27KQNSE26RWSFY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-01T08:10:41.081" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:31.741" grpc.time_ms=6.403 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1938vm-test-run-nixbot-gitlab> gitlab # [ 306.020684] gitaly[1368]: time="2026-07-01T08:10:31.780Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBNJR9DB27KQNSE26RWSFY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:31.779" grpc.time_ms=0.731 pid=1368 span.kind=server system=grpc1939vm-test-run-nixbot-gitlab> gitlab # [ 306.030414] gitaly[1368]: time="2026-07-01T08:10:31.801Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBNJR9DB27KQNSE26RWSFY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:31.800" grpc.time_ms=0.478 pid=1368 span.kind=server system=grpc1940vm-test-run-nixbot-gitlab> gitlab # [ 306.040081] gitaly[1368]: time="2026-07-01T08:10:31.806Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNJR9DB27KQNSE26RWSFY diskcache=26a3d7da-5777-4daf-a06c-6f1b033d6512 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-01T08:10:41.089" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:31.750" pid=1368 ref=refs/keep-around/6d53dbc4843491d6937d6e9373e2dd74d6ee8e92 remote_ip="2001:db8:1::1" revision=6d53dbc4843491d6937d6e9373e2dd74d6ee8e92 span.kind=server system=grpc username=root1941vm-test-run-nixbot-gitlab> gitlab # [ 306.051506] gitaly[1368]: time="2026-07-01T08:10:31.806Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=49 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4984 command.oublock=8 command.real_time_ms=51 command.spawn_token_fork_ms=0 command.system_time_ms=26 command.user_time_ms=23 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNJR9DB27KQNSE26RWSFY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-01T08:10:41.089" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:31.750" grpc.time_ms=56.555 pid=1368 ref=refs/keep-around/6d53dbc4843491d6937d6e9373e2dd74d6ee8e92 remote_ip="2001:db8:1::1" revision=6d53dbc4843491d6937d6e9373e2dd74d6ee8e92 span.kind=server system=grpc username=root1942vm-test-run-nixbot-gitlab> gitlab # [ 306.066691] gitaly[1368]: time="2026-07-01T08:10:31.809Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNJR9DB27KQNSE26RWSFY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:01.609" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:31.809" grpc.time_ms=0.244 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1943vm-test-run-nixbot-gitlab> gitlab # [ 306.108882] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:31 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6d53dbc4843491d6937d6e9373e2dd74d6ee8e92 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 4171944vm-test-run-nixbot-gitlab> gitlab # [ 306.112251] nixbot[3802]: {"ts": "2026-07-01T08:10:31.981Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6d53dbc4843491d6937d6e9373e2dd74d6ee8e92 \"HTTP/1.1 201 Created\""}1945vm-test-run-nixbot-gitlab> gitlab # [ 306.123741] nixbot[3802]: {"ts": "2026-07-01T08:10:31.993Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-6d53dbc48434-db9341e0", "argv0": "bwrap"}1946vm-test-run-nixbot-gitlab> gitlab # [ 306.292402] systemd[1]: Started Nix Daemon.1947vm-test-run-nixbot-gitlab> gitlab # [ 306.370680] nix-daemon[4090]: accepted connection from pid 4088, user nixbot1948vm-test-run-nixbot-gitlab> gitlab # [ 306.374391] nix-daemon[4090]: accepted connection from pid 4089, user nixbot1949vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=70946e52cb9c0a8a279a04d1c7834a4d85e18e8b'1950vm-test-run-nixbot-gitlab> gitlab # [ 306.529743] nixbot[3802]: INFO: ::ffff:127.0.0.1:45126 - "GET /api/repos/gitlab/root/test-flake/builds?commit=70946e52cb9c0a8a279a04d1c7834a4d85e18e8b HTTP/1.1" 200 OK1951vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=70946e52cb9c0a8a279a04d1c7834a4d85e18e8b', in 0.08 seconds)1952vm-test-run-nixbot-gitlab> []1953vm-test-run-nixbot-gitlab> gitlab # [ 306.566249] gitaly[1368]: time="2026-07-01T08:10:32.436Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=3 command.system_time_ms=16 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-01T08:11:02.213" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:32.413" grpc.time_ms=22.384 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1954vm-test-run-nixbot-gitlab> gitlab # [ 306.608292] gitaly[1368]: time="2026-07-01T08:10:32.458Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNHW2CQ14MB40HPAYR9MX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-01T08:11:02.238" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:32.439" grpc.time_ms=19.056 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1955vm-test-run-nixbot-gitlab> gitlab # [ 306.633773] sidekiq-cluster[2429]: {"severity":"WARN","time":"2026-07-01T08:10:32.475Z","class":"CreatePipelineWorker","correlation_id":"01KWEBNHW2CQ14MB40HPAYR9MX","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"6d53dbc4843491d6937d6e9373e2dd74d6ee8e92","after":"70946e52cb9c0a8a279a04d1c7834a4d85e18e8b","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"70946e52cb9c0a8a279a04d1c7834a4d85e18e8b","ref":"refs/heads/master"},"retry":0}1956vm-test-run-nixbot-gitlab> gitlab # [ 306.644368] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:32.482Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"88389dee2e583a4b5532a58f","created_at":"2026-07-01T08:10:31.310Z","trace_propagation_headers":{"sentry-trace":"e596f5905f534fbaaa19d0fb1c1e5a9f-a77b3d665a174671","baggage":"sentry-trace_id=e596f5905f534fbaaa19d0fb1c1e5a9f,sentry-release=Unknown"},"correlation_id":"01KWEBNHW2CQ14MB40HPAYR9MX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9c1a468fb18824b60346d06799f8b73e42feeffcabc689977a6830df7f6168d8","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:31.314Z","job_size_bytes":467,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-88389dee2e583a4b5532a58f: done: 0.910997 sec","job_status":"done","queue_duration_s":0.256029,"scheduling_latency_s":0.256048,"gitaly_calls":7,"gitaly_duration_s":0.075432,"redis_calls":33,"redis_duration_s":0.005704,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000573,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001163,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002476,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001492,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.798619,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.910997,"completed_at":"2026-07-01T08:10:32.481Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032865,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1957vm-test-run-nixbot-gitlab> gitlab # [ 306.702179] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:32.484Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"2db49591-5523-4068-bf67-7d95ed90549a\"}"],"class":"WebHookWorker","jid":"3ad00d27f467387f83bcd75d","created_at":"2026-07-01T08:10:31.436Z","trace_propagation_headers":{"sentry-trace":"e596f5905f534fbaaa19d0fb1c1e5a9f-a77b3d665a174671","baggage":"sentry-trace_id=e596f5905f534fbaaa19d0fb1c1e5a9f,sentry-release=Unknown"},"correlation_id":"01KWEBNHW2CQ14MB40HPAYR9MX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e8ca76f605e5c21fd9329cedec75da446c80dd6636fc2ab3859e43695a78aa73","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:31.442Z","job_size_bytes":1749,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-3ad00d27f467387f83bcd75d: start","job_status":"start","queue_duration_s":1.041297,"scheduling_latency_s":1.041326}1958vm-test-run-nixbot-gitlab> gitlab # [ 306.723182] nixbot[3802]: INFO: ::ffff:127.0.0.1:45132 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1959vm-test-run-nixbot-gitlab> gitlab # [ 306.738789] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:32.607Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"2db49591-5523-4068-bf67-7d95ed90549a\"}"],"class":"WebHookWorker","jid":"3ad00d27f467387f83bcd75d","created_at":"2026-07-01T08:10:31.436Z","trace_propagation_headers":{"sentry-trace":"e596f5905f534fbaaa19d0fb1c1e5a9f-a77b3d665a174671","baggage":"sentry-trace_id=e596f5905f534fbaaa19d0fb1c1e5a9f,sentry-release=Unknown"},"correlation_id":"01KWEBNHW2CQ14MB40HPAYR9MX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e8ca76f605e5c21fd9329cedec75da446c80dd6636fc2ab3859e43695a78aa73","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:31.442Z","job_size_bytes":1749,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-3ad00d27f467387f83bcd75d: done: 0.123159 sec","job_status":"done","queue_duration_s":1.041297,"scheduling_latency_s":1.041326,"redis_calls":19,"redis_duration_s":0.0056,"redis_read_bytes":31,"redis_write_bytes":5282,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001324,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000624,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3644,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002523,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001129,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.015343019999988883,"cpu_s":0.088971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00115876,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.123159,"completed_at":"2026-07-01T08:10:32.607Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003054,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1960vm-test-run-nixbot-gitlab> gitlab # [ 306.794476] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:32.611Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"336b3156c722070bb5c2b49c","created_at":"2026-07-01T08:10:31.818Z","trace_propagation_headers":{"sentry-trace":"c8c802c0b31d485c9f0dc1778482a16d-7ed1fe57d2cc48e2","baggage":"sentry-trace_id=c8c802c0b31d485c9f0dc1778482a16d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNJR9DB27KQNSE26RWSFY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:31.824Z","job_size_bytes":65,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-336b3156c722070bb5c2b49c: start","job_status":"start","queue_duration_s":0.786523,"scheduling_latency_s":0.78657}1961vm-test-run-nixbot-gitlab> gitlab # [ 306.866888] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:32.736Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"336b3156c722070bb5c2b49c","created_at":"2026-07-01T08:10:31.818Z","trace_propagation_headers":{"sentry-trace":"c8c802c0b31d485c9f0dc1778482a16d-7ed1fe57d2cc48e2","baggage":"sentry-trace_id=c8c802c0b31d485c9f0dc1778482a16d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNJR9DB27KQNSE26RWSFY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:31.824Z","job_size_bytes":65,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-336b3156c722070bb5c2b49c: done: 0.12504 sec","job_status":"done","queue_duration_s":0.786523,"scheduling_latency_s":0.78657,"redis_calls":18,"redis_duration_s":0.014809,"redis_read_bytes":271,"redis_write_bytes":2103,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.009708,"redis_feature_flag_read_bytes":264,"redis_feature_flag_write_bytes":1285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004365,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000736,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00080344,"duration_s":0.12504,"completed_at":"2026-07-01T08:10:32.736Z","load_balancing_strategy":"primary","db_duration_s":0.024948,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1962vm-test-run-nixbot-gitlab> gitlab # [ 306.919145] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:32.743Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f7f0052e46107a99dd6da2e1","created_at":"2026-07-01T08:10:31.825Z","trace_propagation_headers":{"sentry-trace":"c8c802c0b31d485c9f0dc1778482a16d-7ed1fe57d2cc48e2","baggage":"sentry-trace_id=c8c802c0b31d485c9f0dc1778482a16d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNJR9DB27KQNSE26RWSFY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:31.830Z","job_size_bytes":65,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f7f0052e46107a99dd6da2e1: start","job_status":"start","queue_duration_s":0.91383,"scheduling_latency_s":0.91387}1963vm-test-run-nixbot-gitlab> gitlab # [ 306.938940] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:32 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 1351964vm-test-run-nixbot-gitlab> gitlab # [ 306.941638] gitaly[1368]: time="2026-07-01T08:10:32.757Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=29 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNKSENT0B1PZE4HTEWANP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:32.724" grpc.time_ms=32.638 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc1965vm-test-run-nixbot-gitlab> gitlab # [ 307.000381] gitaly[1368]: time="2026-07-01T08:10:32.870Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNKXQ9M35XSZC0A4RSTCK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:32.860" pid=1368 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1966vm-test-run-nixbot-gitlab> gitlab # [ 307.015103] gitaly[1368]: time="2026-07-01T08:10:32.874Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNKXQ9M35XSZC0A4RSTCK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:32.860" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc1967vm-test-run-nixbot-gitlab> gitlab # [ 307.033401] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:32.884Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f7f0052e46107a99dd6da2e1","created_at":"2026-07-01T08:10:31.825Z","trace_propagation_headers":{"sentry-trace":"c8c802c0b31d485c9f0dc1778482a16d-7ed1fe57d2cc48e2","baggage":"sentry-trace_id=c8c802c0b31d485c9f0dc1778482a16d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNJR9DB27KQNSE26RWSFY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:31.830Z","job_size_bytes":65,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f7f0052e46107a99dd6da2e1: done: 0.140284 sec","job_status":"done","queue_duration_s":0.91383,"scheduling_latency_s":0.91387,"redis_calls":21,"redis_duration_s":0.014306,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.006965,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002406,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004935,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.091245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040944,"duration_s":0.140284,"completed_at":"2026-07-01T08:10:32.884Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027676,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1968vm-test-run-nixbot-gitlab> gitlab # [ 307.084647] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:32.887Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"a275731f73752ae2e3fad91d","created_at":"2026-07-01T08:10:31.956Z","trace_propagation_headers":{"sentry-trace":"c8c802c0b31d485c9f0dc1778482a16d-7ed1fe57d2cc48e2","baggage":"sentry-trace_id=c8c802c0b31d485c9f0dc1778482a16d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNJR9DB27KQNSE26RWSFY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:31.962Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a275731f73752ae2e3fad91d: start","job_status":"start","queue_duration_s":0.925261,"scheduling_latency_s":0.925295}1969vm-test-run-nixbot-gitlab> gitlab # [ 307.101414] gitlab-workhorse[1300]: time="2026-07-01T08:10:32Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671970vm-test-run-nixbot-gitlab> gitlab # [ 307.103314] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:32 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 1221971vm-test-run-nixbot-gitlab> gitlab # [ 307.105301] gitaly[1368]: time="2026-07-01T08:10:32.874Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNKXQ9M35XSZC0A4RSTCK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:32.860" grpc.time_ms=14.283 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc1972vm-test-run-nixbot-gitlab> gitlab # [ 307.120202] nixbot[3802]: {"ts": "2026-07-01T08:10:32.918Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"}1973vm-test-run-nixbot-gitlab> gitlab # [ 307.122337] nix-daemon[4095]: unexpected Nix daemon error: error: interrupted by the user1974vm-test-run-nixbot-gitlab> gitlab # [ 307.148081] gitaly[1368]: time="2026-07-01T08:10:33.015Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNJR9DB27KQNSE26RWSFY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-01T08:11:03.013" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:33.013" grpc.time_ms=1.687 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1975vm-test-run-nixbot-gitlab> gitlab # [ 307.160904] gitaly[1368]: time="2026-07-01T08:10:33.026Z" level=info msg="finished unary call with code OK" catfile.duration_ms=6 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=6 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNM2VAFVZF5PNQ0VMDW8N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:03.014" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:33.014" grpc.time_ms=11.843 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1976vm-test-run-nixbot-gitlab> gitlab # [ 307.179621] gitaly[1368]: time="2026-07-01T08:10:33.049Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=593 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNM2VAFVZF5PNQ0VMDW8N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-01T08:11:03.039" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:33.040" grpc.time_ms=9.493 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1977vm-test-run-nixbot-gitlab> gitlab # [ 307.195049] gitaly[1368]: time="2026-07-01T08:10:33.065Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNJR9DB27KQNSE26RWSFY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:03.064" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:33.064" grpc.time_ms=0.651 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1978vm-test-run-nixbot-gitlab> gitlab # [ 307.211179] gitaly[1368]: time="2026-07-01T08:10:33.070Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNM4M1NNVCVX8212GA0M1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:03.068" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:33.068" grpc.time_ms=2.052 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1979vm-test-run-nixbot-gitlab> gitlab # [ 307.224468] gitaly[1368]: time="2026-07-01T08:10:33.074Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNM2VAFVZF5PNQ0VMDW8N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T08:10:43.012" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:33.062" grpc.time_ms=11.809 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1980vm-test-run-nixbot-gitlab> gitlab # [ 307.238125] gitaly[1368]: time="2026-07-01T08:10:33.097Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNJR9DB27KQNSE26RWSFY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:03.096" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:33.096" grpc.time_ms=0.473 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1981vm-test-run-nixbot-gitlab> gitlab # [ 307.250621] gitaly[1368]: time="2026-07-01T08:10:33.099Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNM2VAFVZF5PNQ0VMDW8N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-01T08:10:43.048" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:33.099" grpc.time_ms=0.218 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1982vm-test-run-nixbot-gitlab> gitlab # [ 307.262235] gitaly[1368]: time="2026-07-01T08:10:33.114Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNM2VAFVZF5PNQ0VMDW8N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-01T08:10:43.053" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:33.103" grpc.time_ms=10.555 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1983vm-test-run-nixbot-gitlab> gitlab # [ 307.275878] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:33.109Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"a275731f73752ae2e3fad91d","created_at":"2026-07-01T08:10:31.956Z","trace_propagation_headers":{"sentry-trace":"c8c802c0b31d485c9f0dc1778482a16d-7ed1fe57d2cc48e2","baggage":"sentry-trace_id=c8c802c0b31d485c9f0dc1778482a16d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNJR9DB27KQNSE26RWSFY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:31.962Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a275731f73752ae2e3fad91d: done: 0.222087 sec","job_status":"done","queue_duration_s":0.925261,"scheduling_latency_s":0.925295,"gitaly_calls":3,"gitaly_duration_s":0.009605,"redis_calls":48,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.018906,"redis_read_bytes":208,"redis_write_bytes":6800,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.004569,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001336,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.002287,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.000289,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1130,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.00174,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.00285,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000184,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001082,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.006,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.025,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.153479,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.18346766,"duration_s":0.222087,"completed_at":"2026-07-01T08:10:33.109Z","load_balancing_strategy":"primary","db_duration_s":0.051547,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1984vm-test-run-nixbot-gitlab> gitlab # [ 307.331499] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:33.111Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","3ad00d27f467387f83bcd75d"],"class":"WebHooks::LogExecutionWorker","jid":"1476ef243adf680b7ec7bf5c","created_at":"2026-07-01T08:10:32.579Z","trace_propagation_headers":{"sentry-trace":"e596f5905f534fbaaa19d0fb1c1e5a9f-89571f1662ca4670","baggage":"sentry-trace_id=e596f5905f534fbaaa19d0fb1c1e5a9f,sentry-release=Unknown"},"correlation_id":"01KWEBNHW2CQ14MB40HPAYR9MX","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ca3fbf197a2f012e9a4e6e187c5276516503eb1bf5eda1b391514e91a6bc7c37","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:32.587Z","job_size_bytes":2507,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-1476ef243adf680b7ec7bf5c: start","job_status":"start","queue_duration_s":0.523717,"scheduling_latency_s":0.523746}1985vm-test-run-nixbot-gitlab> gitlab # [ 307.350482] gitaly[1368]: time="2026-07-01T08:10:33.155Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBNM2VAFVZF5PNQ0VMDW8N grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:33.155" grpc.time_ms=0.343 pid=1368 span.kind=server system=grpc1986vm-test-run-nixbot-gitlab> gitlab # [ 307.364794] gitaly[1368]: time="2026-07-01T08:10:33.190Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBNM2VAFVZF5PNQ0VMDW8N grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:33.189" grpc.time_ms=0.378 pid=1368 span.kind=server system=grpc1987vm-test-run-nixbot-gitlab> gitlab # [ 307.374653] gitaly[1368]: time="2026-07-01T08:10:33.198Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=7 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNM4M1NNVCVX8212GA0M1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T08:10:43.115" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:33.185" grpc.time_ms=12.752 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1988vm-test-run-nixbot-gitlab> gitlab # [ 307.388546] gitaly[1368]: time="2026-07-01T08:10:33.199Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNM2VAFVZF5PNQ0VMDW8N diskcache=135c085f-df73-4007-8fcf-d8834b03ac1a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-01T08:10:43.065" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:33.115" pid=1368 ref=refs/keep-around/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b remote_ip="2001:db8:1::1" revision=70946e52cb9c0a8a279a04d1c7834a4d85e18e8b span.kind=server system=grpc username=root1989vm-test-run-nixbot-gitlab> gitlab # [ 307.399982] gitaly[1368]: time="2026-07-01T08:10:33.199Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=48 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5124 command.oublock=8 command.real_time_ms=75 command.spawn_token_fork_ms=3 command.system_time_ms=30 command.user_time_ms=18 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNM2VAFVZF5PNQ0VMDW8N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-01T08:10:43.065" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T08:10:33.115" grpc.time_ms=83.585 pid=1368 ref=refs/keep-around/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b remote_ip="2001:db8:1::1" revision=70946e52cb9c0a8a279a04d1c7834a4d85e18e8b span.kind=server system=grpc username=root1990vm-test-run-nixbot-gitlab> gitlab # [ 307.414998] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:33.260Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","3ad00d27f467387f83bcd75d"],"class":"WebHooks::LogExecutionWorker","jid":"1476ef243adf680b7ec7bf5c","created_at":"2026-07-01T08:10:32.579Z","trace_propagation_headers":{"sentry-trace":"e596f5905f534fbaaa19d0fb1c1e5a9f-89571f1662ca4670","baggage":"sentry-trace_id=e596f5905f534fbaaa19d0fb1c1e5a9f,sentry-release=Unknown"},"correlation_id":"01KWEBNHW2CQ14MB40HPAYR9MX","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ca3fbf197a2f012e9a4e6e187c5276516503eb1bf5eda1b391514e91a6bc7c37","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:32.587Z","job_size_bytes":2507,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-1476ef243adf680b7ec7bf5c: done: 0.14935 sec","job_status":"done","queue_duration_s":0.523717,"scheduling_latency_s":0.523746,"redis_calls":10,"redis_duration_s":0.005103,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000292,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004313,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000498,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.035,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.035,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.130807,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00224176,"duration_s":0.14935,"completed_at":"2026-07-01T08:10:33.260Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007389,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1991vm-test-run-nixbot-gitlab> gitlab # [ 307.460579] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:33.263Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"7e2980f1c8cb4c380b9a0087","created_at":"2026-07-01T08:10:32.990Z","trace_propagation_headers":{"sentry-trace":"c8c802c0b31d485c9f0dc1778482a16d-4880923802d54679","baggage":"sentry-trace_id=c8c802c0b31d485c9f0dc1778482a16d,sentry-release=Unknown"},"correlation_id":"01KWEBNJR9DB27KQNSE26RWSFY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:32.997Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7e2980f1c8cb4c380b9a0087: start","job_status":"start","queue_duration_s":0.266241,"scheduling_latency_s":0.266265}1992vm-test-run-nixbot-gitlab> gitlab # [ 307.476959] nixbot[3802]: {"ts": "2026-07-01T08:10:33.281Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b \"HTTP/1.1 201 Created\""}1993vm-test-run-nixbot-gitlab> gitlab # [ 307.479673] nixbot[3802]: {"ts": "2026-07-01T08:10:33.291Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-70946e52cb9c-69ed9a5d", "argv0": "bwrap"}1994vm-test-run-nixbot-gitlab> gitlab # [ 307.482236] gitaly[1368]: time="2026-07-01T08:10:33.204Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNM2VAFVZF5PNQ0VMDW8N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:03.002" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:33.204" grpc.time_ms=0.31 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1995vm-test-run-nixbot-gitlab> gitlab # [ 307.495271] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:33 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 3551996vm-test-run-nixbot-gitlab> gitlab # [ 307.497988] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:33 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6d53dbc4843491d6937d6e9373e2dd74d6ee8e92 HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 3331997vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=70946e52cb9c0a8a279a04d1c7834a4d85e18e8b'1998vm-test-run-nixbot-gitlab> gitlab # [ 307.500582] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:33.307Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"7e2980f1c8cb4c380b9a0087","created_at":"2026-07-01T08:10:32.990Z","trace_propagation_headers":{"sentry-trace":"c8c802c0b31d485c9f0dc1778482a16d-4880923802d54679","baggage":"sentry-trace_id=c8c802c0b31d485c9f0dc1778482a16d,sentry-release=Unknown"},"correlation_id":"01KWEBNJR9DB27KQNSE26RWSFY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:32.997Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7e2980f1c8cb4c380b9a0087: done: 0.044561 sec","job_status":"done","queue_duration_s":0.266241,"scheduling_latency_s":0.266265,"redis_calls":10,"redis_duration_s":0.005411,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003019,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001658,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000734,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00021784,"duration_s":0.044561,"completed_at":"2026-07-01T08:10:33.307Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007429,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1999vm-test-run-nixbot-gitlab> gitlab # [ 307.548331] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:33.309Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"9758cd832b88c2d9b1bb06ab","created_at":"2026-07-01T08:10:33.206Z","trace_propagation_headers":{"sentry-trace":"4a3a3620c3654e7e8b198d381c1e3125-342a32befe8444ef","baggage":"sentry-trace_id=4a3a3620c3654e7e8b198d381c1e3125,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNM2VAFVZF5PNQ0VMDW8N","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:33.209Z","job_size_bytes":65,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-9758cd832b88c2d9b1bb06ab: start","job_status":"start","queue_duration_s":0.099707,"scheduling_latency_s":0.099728}2000vm-test-run-nixbot-gitlab> gitlab # [ 307.565503] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:33.332Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"9758cd832b88c2d9b1bb06ab","created_at":"2026-07-01T08:10:33.206Z","trace_propagation_headers":{"sentry-trace":"4a3a3620c3654e7e8b198d381c1e3125-342a32befe8444ef","baggage":"sentry-trace_id=4a3a3620c3654e7e8b198d381c1e3125,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNM2VAFVZF5PNQ0VMDW8N","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:33.209Z","job_size_bytes":65,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-9758cd832b88c2d9b1bb06ab: done: 0.02354 sec","job_status":"done","queue_duration_s":0.099707,"scheduling_latency_s":0.099728,"redis_calls":7,"redis_duration_s":0.0011819999999999999,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000299,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02354,"completed_at":"2026-07-01T08:10:33.332Z","load_balancing_strategy":"primary","db_duration_s":0.002775,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2001vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=70946e52cb9c0a8a279a04d1c7834a4d85e18e8b', in 0.10 seconds)2002vm-test-run-nixbot-gitlab> [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '70946e52cb9c0a8a279a04d1c7834a4d85e18e8b', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-07-01T08:10:32.907932Z', 'started_at': '2026-07-01T08:10:33.282896Z', 'finished_at': None}]2003vm-test-run-nixbot-gitlab> (finished: subtest: push delivers the webhook, in 3.16 seconds)2004vm-test-run-nixbot-gitlab> subtest: eval and build post commit statuses2005vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b/statuses2006vm-test-run-nixbot-gitlab> gitlab # [ 307.615624] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:33.334Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"3f3affb6886ffe8188ef5965","created_at":"2026-07-01T08:10:33.210Z","trace_propagation_headers":{"sentry-trace":"4a3a3620c3654e7e8b198d381c1e3125-342a32befe8444ef","baggage":"sentry-trace_id=4a3a3620c3654e7e8b198d381c1e3125,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNM2VAFVZF5PNQ0VMDW8N","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:33.212Z","job_size_bytes":65,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-3f3affb6886ffe8188ef5965: start","job_status":"start","queue_duration_s":0.121813,"scheduling_latency_s":0.121834}2007vm-test-run-nixbot-gitlab> gitlab # [ 307.635800] nixbot[3802]: {"ts": "2026-07-01T08:10:33.352Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6d53dbc4843491d6937d6e9373e2dd74d6ee8e92 \"HTTP/1.1 201 Created\""}2008vm-test-run-nixbot-gitlab> gitlab # [ 307.638937] nixbot[3802]: INFO: ::ffff:127.0.0.1:45140 - "GET /api/repos/gitlab/root/test-flake/builds?commit=70946e52cb9c0a8a279a04d1c7834a4d85e18e8b HTTP/1.1" 200 OK2009vm-test-run-nixbot-gitlab> gitlab # [ 307.644588] nix-daemon[4090]: accepted connection from pid 4167, user nixbot2010vm-test-run-nixbot-gitlab> gitlab # [ 307.645603] nix-daemon[4090]: accepted connection from pid 4165, user nixbot2011vm-test-run-nixbot-gitlab> gitlab # [ 307.646680] gitaly[1368]: time="2026-07-01T08:10:33.415Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNMGFXC5NTJ3602VE0BPN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:03.213" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:33.414" grpc.time_ms=0.842 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2012vm-test-run-nixbot-gitlab> gitlab # [ 307.660424] nixbot[3802]: {"ts": "2026-07-01T08:10:33.512Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6d53dbc4843491d6937d6e9373e2dd74d6ee8e92 \"HTTP/1.1 201 Created\""}2013vm-test-run-nixbot-gitlab> gitlab # [ 307.663517] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:33 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6d53dbc4843491d6937d6e9373e2dd74d6ee8e92 HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 1512014vm-test-run-nixbot-gitlab> gitlab # [ 307.666304] gitaly[1368]: time="2026-07-01T08:10:33.455Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNMGFXC5NTJ3602VE0BPN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T08:10:43.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:33.448" grpc.time_ms=7.661 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2015vm-test-run-nixbot-gitlab> gitlab # [ 307.680440] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:33.533Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"3f3affb6886ffe8188ef5965","created_at":"2026-07-01T08:10:33.210Z","trace_propagation_headers":{"sentry-trace":"4a3a3620c3654e7e8b198d381c1e3125-342a32befe8444ef","baggage":"sentry-trace_id=4a3a3620c3654e7e8b198d381c1e3125,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNM2VAFVZF5PNQ0VMDW8N","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:33.212Z","job_size_bytes":65,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-3f3affb6886ffe8188ef5965: done: 0.199675 sec","job_status":"done","queue_duration_s":0.121813,"scheduling_latency_s":0.121834,"redis_calls":10,"redis_duration_s":0.001689,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000916,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000773,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.153,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.145,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.199675,"completed_at":"2026-07-01T08:10:33.533Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.15966,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2016vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b/statuses, in 0.11 seconds)2017vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '70946e52cb9c0a8a279a04d1c7834a4d85e18e8b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-01T08:10:33.255Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2018vm-test-run-nixbot-gitlab> gitlab # [ 307.726130] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:33.535Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"074010b9ec6911cd1173bce4","created_at":"2026-07-01T08:10:33.261Z","trace_propagation_headers":{"sentry-trace":"4a3a3620c3654e7e8b198d381c1e3125-342a32befe8444ef","baggage":"sentry-trace_id=4a3a3620c3654e7e8b198d381c1e3125,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNM2VAFVZF5PNQ0VMDW8N","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:33.263Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-074010b9ec6911cd1173bce4: start","job_status":"start","queue_duration_s":0.271343,"scheduling_latency_s":0.271365}2019vm-test-run-nixbot-gitlab> gitlab # [ 307.742464] gitaly[1368]: time="2026-07-01T08:10:33.575Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNMNRPKT4EBXRBYPFA5D4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:03.572" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:33.573" grpc.time_ms=1.593 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2020vm-test-run-nixbot-gitlab> gitlab # [ 307.755422] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:33 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 762021vm-test-run-nixbot-gitlab> gitlab # [ 307.758286] gitaly[1368]: time="2026-07-01T08:10:33.603Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNM2VAFVZF5PNQ0VMDW8N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-01T08:11:03.601" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:33.601" grpc.time_ms=1.324 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2022vm-test-run-nixbot-gitlab> gitlab # [ 307.772697] gitaly[1368]: time="2026-07-01T08:10:33.637Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNM2VAFVZF5PNQ0VMDW8N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:03.635" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:33.635" grpc.time_ms=1.232 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2023vm-test-run-nixbot-gitlab> gitlab # [ 307.797324] gitaly[1368]: time="2026-07-01T08:10:33.667Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNM2VAFVZF5PNQ0VMDW8N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:03.665" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:33.666" grpc.time_ms=0.808 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2024vm-test-run-nixbot-gitlab> gitlab # [ 307.810460] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:33.678Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"074010b9ec6911cd1173bce4","created_at":"2026-07-01T08:10:33.261Z","trace_propagation_headers":{"sentry-trace":"4a3a3620c3654e7e8b198d381c1e3125-342a32befe8444ef","baggage":"sentry-trace_id=4a3a3620c3654e7e8b198d381c1e3125,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNM2VAFVZF5PNQ0VMDW8N","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:33.263Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-074010b9ec6911cd1173bce4: done: 0.143212 sec","job_status":"done","queue_duration_s":0.271343,"scheduling_latency_s":0.271365,"gitaly_calls":3,"gitaly_duration_s":0.011025,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.005695,"redis_read_bytes":98,"redis_write_bytes":6014,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000698,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001642,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000199,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001184,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000183,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000312,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000779,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.102436,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.12103746,"duration_s":0.143212,"completed_at":"2026-07-01T08:10:33.678Z","load_balancing_strategy":"primary","db_duration_s":0.031876,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2025vm-test-run-nixbot-gitlab> gitlab # [ 307.862561] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:33.679Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"45ed2807db7c2aaa6d2e087a","created_at":"2026-07-01T08:10:33.289Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-5c84b9332daf43ad","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:33.296Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-45ed2807db7c2aaa6d2e087a: start","job_status":"start","queue_duration_s":0.382764,"scheduling_latency_s":0.382784}2026vm-test-run-nixbot-gitlab> gitlab # [ 307.972496] gitaly[1368]: time="2026-07-01T08:10:33.841Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBNM4M1NNVCVX8212GA0M1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:33.840" grpc.time_ms=0.882 pid=1368 span.kind=server system=grpc2027vm-test-run-nixbot-gitlab> gitlab # [ 308.020281] gitaly[1368]: time="2026-07-01T08:10:33.890Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBNM4M1NNVCVX8212GA0M1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:33.889" grpc.time_ms=0.872 pid=1368 span.kind=server system=grpc2028vm-test-run-nixbot-gitlab> gitlab # [ 308.069147] gitaly[1368]: time="2026-07-01T08:10:33.939Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBNM4M1NNVCVX8212GA0M1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:33.938" grpc.time_ms=0.844 pid=1368 span.kind=server system=grpc2029vm-test-run-nixbot-gitlab> gitlab # [ 308.095748] gitaly[1368]: time="2026-07-01T08:10:33.951Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNM4M1NNVCVX8212GA0M1 diskcache=b0599c32-5c10-4ec7-8200-74c3027c45c6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-01T08:11:03.788" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:33.788" pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2030vm-test-run-nixbot-gitlab> gitlab # [ 308.115633] gitaly[1368]: time="2026-07-01T08:10:33.952Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=179 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7176 command.oublock=0 command.real_time_ms=155 command.spawn_token_fork_ms=0 command.system_time_ms=97 command.user_time_ms=82 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNM4M1NNVCVX8212GA0M1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-01T08:11:03.788" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:33.788" grpc.time_ms=163.127 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2031vm-test-run-nixbot-gitlab> gitlab # [ 308.354409] gitaly[1368]: time="2026-07-01T08:10:34.224Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNM4M1NNVCVX8212GA0M1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-01T08:11:04.222" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:34.222" grpc.time_ms=1.581 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2032vm-test-run-nixbot-gitlab> gitlab # [ 308.424948] gitaly[1368]: time="2026-07-01T08:10:34.295Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNM4M1NNVCVX8212GA0M1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:04.292" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:34.293" grpc.time_ms=1.319 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2033vm-test-run-nixbot-gitlab> gitlab # [ 308.519389] gitaly[1368]: time="2026-07-01T08:10:34.389Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNM4M1NNVCVX8212GA0M1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:04.387" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:34.387" grpc.time_ms=1.335 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2034vm-test-run-nixbot-gitlab> gitlab # [ 308.550661] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:34.408Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"45ed2807db7c2aaa6d2e087a","created_at":"2026-07-01T08:10:33.289Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-5c84b9332daf43ad","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:33.296Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2429,"job_status":"rescheduled","message":"PipelineProcessWorker JID-45ed2807db7c2aaa6d2e087a: rescheduled"}2035vm-test-run-nixbot-gitlab> gitlab # [ 308.576034] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:34.425Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"45ed2807db7c2aaa6d2e087a","created_at":"2026-07-01T08:10:33.289Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-5c84b9332daf43ad","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:33.296Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-45ed2807db7c2aaa6d2e087a: done: 0.745673 sec","job_status":"done","queue_duration_s":0.382764,"scheduling_latency_s":0.382784,"gitaly_calls":4,"gitaly_duration_s":0.183106,"redis_calls":101,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.032198,"redis_read_bytes":185,"redis_write_bytes":23552,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001269,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003298,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.008919,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.004973,"redis_queues_read_bytes":25,"redis_queues_write_bytes":12384,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.00833,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000298,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000858,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002984,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.084,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.022,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.024,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.046,"db_ci_duration_s":0.038,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.471142,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.69708896,"duration_s":0.745673,"completed_at":"2026-07-01T08:10:34.425Z","load_balancing_strategy":"primary","db_duration_s":0.090533,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2036vm-test-run-nixbot-gitlab> gitlab # [ 308.644663] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:34.428Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"435b7c36ff996f42c7f5f481","created_at":"2026-07-01T08:10:33.586Z","trace_propagation_headers":{"sentry-trace":"4a3a3620c3654e7e8b198d381c1e3125-2cf403044a364115","baggage":"sentry-trace_id=4a3a3620c3654e7e8b198d381c1e3125,sentry-release=Unknown"},"correlation_id":"01KWEBNM2VAFVZF5PNQ0VMDW8N","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:33.588Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-435b7c36ff996f42c7f5f481: start","job_status":"start","queue_duration_s":0.840435,"scheduling_latency_s":0.840481}2037vm-test-run-nixbot-gitlab> gitlab # [ 308.661680] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:34.487Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"435b7c36ff996f42c7f5f481","created_at":"2026-07-01T08:10:33.586Z","trace_propagation_headers":{"sentry-trace":"4a3a3620c3654e7e8b198d381c1e3125-2cf403044a364115","baggage":"sentry-trace_id=4a3a3620c3654e7e8b198d381c1e3125,sentry-release=Unknown"},"correlation_id":"01KWEBNM2VAFVZF5PNQ0VMDW8N","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:33.588Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-435b7c36ff996f42c7f5f481: done: 0.058534 sec","job_status":"done","queue_duration_s":0.840435,"scheduling_latency_s":0.840481,"redis_calls":7,"redis_duration_s":0.002823,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002013,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00081,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042378,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.058534,"completed_at":"2026-07-01T08:10:34.487Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006907,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2038vm-test-run-nixbot-gitlab> gitlab # [ 308.704822] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:34.489Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"5a5c86b886eb2e086fc889d9","created_at":"2026-07-01T08:10:33.759Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:33.765Z","job_size_bytes":23,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-5a5c86b886eb2e086fc889d9: start","job_status":"start","queue_duration_s":0.724537,"scheduling_latency_s":0.72457}2039vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b/statuses2040vm-test-run-nixbot-gitlab> gitlab # [ 308.781699] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:34.651Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"5a5c86b886eb2e086fc889d9","created_at":"2026-07-01T08:10:33.759Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:33.765Z","job_size_bytes":23,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-5a5c86b886eb2e086fc889d9: done: 0.161476 sec","job_status":"done","queue_duration_s":0.724537,"scheduling_latency_s":0.72457,"redis_calls":15,"redis_duration_s":0.005613,"redis_read_bytes":13,"redis_write_bytes":3011,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000611,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.001402,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1626,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00226,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.00069,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00065,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.127867,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00054238,"duration_s":0.161476,"completed_at":"2026-07-01T08:10:34.651Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022644,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2041vm-test-run-nixbot-gitlab> gitlab # [ 308.859654] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:34.653Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"792698e17f4c9921843a7476","created_at":"2026-07-01T08:10:33.961Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:33.972Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-792698e17f4c9921843a7476: start","job_status":"start","queue_duration_s":0.6803,"scheduling_latency_s":0.680322}2042vm-test-run-nixbot-gitlab> gitlab # [ 308.877839] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:34.698Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"792698e17f4c9921843a7476","created_at":"2026-07-01T08:10:33.961Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:33.972Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-792698e17f4c9921843a7476: done: 0.045139 sec","job_status":"done","queue_duration_s":0.6803,"scheduling_latency_s":0.680322,"redis_calls":10,"redis_duration_s":0.005941,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000548,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002371,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003022,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00046468,"duration_s":0.045139,"completed_at":"2026-07-01T08:10:34.698Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01106,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2043vm-test-run-nixbot-gitlab> gitlab # [ 308.922813] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:34.700Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"8fd5698c7a954568ee788e09","created_at":"2026-07-01T08:10:33.997Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.009Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-8fd5698c7a954568ee788e09: start","job_status":"start","queue_duration_s":0.690894,"scheduling_latency_s":0.690915}2044vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b/statuses, in 0.22 seconds)2045vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '70946e52cb9c0a8a279a04d1c7834a4d85e18e8b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-01T08:10:33.255Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2046vm-test-run-nixbot-gitlab> gitlab # [ 308.939068] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:34.752Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"8fd5698c7a954568ee788e09","created_at":"2026-07-01T08:10:33.997Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.009Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-8fd5698c7a954568ee788e09: done: 0.05268 sec","job_status":"done","queue_duration_s":0.690894,"scheduling_latency_s":0.690915,"redis_calls":10,"redis_duration_s":0.002282,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000507,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001322,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000453,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030152,"duration_s":0.05268,"completed_at":"2026-07-01T08:10:34.752Z","load_balancing_strategy":"primary","db_duration_s":0.017629,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2047vm-test-run-nixbot-gitlab> gitlab # [ 308.983027] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:34.754Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"f1e702bce508d2978a2162fb","created_at":"2026-07-01T08:10:34.637Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-6b4dd7f5b9034e50","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:4de9cc7af18565cd10460818456f5d11346a662afe720558533b51f3132b2c18","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.639Z","job_size_bytes":464,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-f1e702bce508d2978a2162fb: start","job_status":"start","queue_duration_s":0.114362,"scheduling_latency_s":0.114382}2048vm-test-run-nixbot-gitlab> gitlab # [ 309.000196] sidekiq-cluster[2429]: {"severity":"WARN","time":"2026-07-01T08:10:34.802Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"f1e702bce508d2978a2162fb","created_at":"2026-07-01T08:10:34.637Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-6b4dd7f5b9034e50","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:4de9cc7af18565cd10460818456f5d11346a662afe720558533b51f3132b2c18","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.639Z","job_size_bytes":464,"pid":2429,"sidekiq_tid":"m4et","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-f1e702bce508d2978a2162fb: fail: 0.04739 sec","job_status":"fail","queue_duration_s":0.114362,"scheduling_latency_s":0.114382,"redis_calls":7,"redis_duration_s":0.0010470000000000002,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000756,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000291,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04739,"completed_at":"2026-07-01T08:10:34.801Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/w9mp1gyfs2qdv12r0ji70idzndmxwz2s-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/p8ylyr0sj9sxcknqcpja13nypqixk2dp-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/dndr7b4a17339dsm67vzb6nv4p2kp82q-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/zx6ndpscxs64hnidz69ab430ccms6rak-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.008909,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2049vm-test-run-nixbot-gitlab> gitlab # [ 309.385233] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:34.805Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"91a0d5760e789c2d65fd6e4e","created_at":"2026-07-01T08:10:34.035Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.044Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-91a0d5760e789c2d65fd6e4e: start","job_status":"start","queue_duration_s":0.76134,"scheduling_latency_s":0.761388}2050vm-test-run-nixbot-gitlab> gitlab # [ 309.400978] gitaly[1368]: time="2026-07-01T08:10:34.771Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNNSQWRPKANZRATPWQSG0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:04.768" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:34.769" grpc.time_ms=1.456 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2051vm-test-run-nixbot-gitlab> gitlab # [ 309.414948] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1432052vm-test-run-nixbot-gitlab> gitlab # [ 309.417754] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:34.887Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"91a0d5760e789c2d65fd6e4e","created_at":"2026-07-01T08:10:34.035Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.044Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-91a0d5760e789c2d65fd6e4e: done: 0.081707 sec","job_status":"done","queue_duration_s":0.76134,"scheduling_latency_s":0.761388,"redis_calls":13,"redis_duration_s":0.006259,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000593,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004053,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000892,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000721,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00061398,"duration_s":0.081707,"completed_at":"2026-07-01T08:10:34.886Z","load_balancing_strategy":"primary","db_duration_s":0.008119,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2053vm-test-run-nixbot-gitlab> gitlab # [ 309.463327] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:34.889Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6a0cb357d7dca979c51724f6","created_at":"2026-07-01T08:10:34.045Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.053Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6a0cb357d7dca979c51724f6: start","job_status":"start","queue_duration_s":0.836375,"scheduling_latency_s":0.836405}2054vm-test-run-nixbot-gitlab> gitlab # [ 309.480219] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:34.941Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6a0cb357d7dca979c51724f6","created_at":"2026-07-01T08:10:34.045Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.053Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6a0cb357d7dca979c51724f6: done: 0.052079 sec","job_status":"done","queue_duration_s":0.836375,"scheduling_latency_s":0.836405,"redis_calls":10,"redis_duration_s":0.003771,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000555,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00256,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000656,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00051172,"duration_s":0.052079,"completed_at":"2026-07-01T08:10:34.941Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008606,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2055vm-test-run-nixbot-gitlab> gitlab # [ 309.525721] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:34.943Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"14bde99e99910920668fc852","created_at":"2026-07-01T08:10:34.054Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.062Z","job_size_bytes":19,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-14bde99e99910920668fc852: start","job_status":"start","queue_duration_s":0.880672,"scheduling_latency_s":0.880698}2056vm-test-run-nixbot-gitlab> gitlab # [ 309.573922] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:34.990Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"14bde99e99910920668fc852","created_at":"2026-07-01T08:10:34.054Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.062Z","job_size_bytes":19,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-14bde99e99910920668fc852: done: 0.046998 sec","job_status":"done","queue_duration_s":0.880672,"scheduling_latency_s":0.880698,"redis_calls":11,"redis_duration_s":0.002999,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000547,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001908,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000544,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027169,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00041148,"duration_s":0.046998,"completed_at":"2026-07-01T08:10:34.990Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007853,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2057vm-test-run-nixbot-gitlab> gitlab # [ 309.650581] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:34.992Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"a46ae90fca1efbf2bd79a42a","created_at":"2026-07-01T08:10:34.067Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.212Z","job_size_bytes":65,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-a46ae90fca1efbf2bd79a42a: start","job_status":"start","queue_duration_s":0.779786,"scheduling_latency_s":0.779808}2058vm-test-run-nixbot-gitlab> gitlab # [ 309.675748] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:35.033Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"a46ae90fca1efbf2bd79a42a","created_at":"2026-07-01T08:10:34.067Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.212Z","job_size_bytes":65,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-a46ae90fca1efbf2bd79a42a: done: 0.041691 sec","job_status":"done","queue_duration_s":0.779786,"scheduling_latency_s":0.779808,"redis_calls":10,"redis_duration_s":0.0030629999999999998,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000509,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001947,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000607,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00054288,"duration_s":0.041691,"completed_at":"2026-07-01T08:10:35.033Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006735,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2059vm-test-run-nixbot-gitlab> gitlab # [ 309.729693] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:35.035Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"b86f6037cd50f33da296f6d5","created_at":"2026-07-01T08:10:34.213Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.215Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b86f6037cd50f33da296f6d5: start","job_status":"start","queue_duration_s":0.820254,"scheduling_latency_s":0.820275}2060vm-test-run-nixbot-gitlab> gitlab # [ 309.746015] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:35.070Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"b86f6037cd50f33da296f6d5","created_at":"2026-07-01T08:10:34.213Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.215Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b86f6037cd50f33da296f6d5: done: 0.034788 sec","job_status":"done","queue_duration_s":0.820254,"scheduling_latency_s":0.820275,"redis_calls":7,"redis_duration_s":0.001897,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001363,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000534,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034788,"completed_at":"2026-07-01T08:10:35.070Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005854,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2061vm-test-run-nixbot-gitlab> gitlab # [ 309.788929] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:35.072Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"d9f4cd28e6bb14aaedd92fde","created_at":"2026-07-01T08:10:34.297Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.308Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-d9f4cd28e6bb14aaedd92fde: start","job_status":"start","queue_duration_s":0.763163,"scheduling_latency_s":0.763183}2062vm-test-run-nixbot-gitlab> gitlab # [ 309.805459] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:35.116Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"d9f4cd28e6bb14aaedd92fde","created_at":"2026-07-01T08:10:34.297Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.308Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-d9f4cd28e6bb14aaedd92fde: done: 0.043938 sec","job_status":"done","queue_duration_s":0.763163,"scheduling_latency_s":0.763183,"redis_calls":10,"redis_duration_s":0.002632,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000463,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001629,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00054,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00048132,"duration_s":0.043938,"completed_at":"2026-07-01T08:10:35.115Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014904,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2063vm-test-run-nixbot-gitlab> gitlab # [ 309.851039] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:35.119Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"f2cb7ec2edf3721f86331b75","created_at":"2026-07-01T08:10:34.310Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.321Z","job_size_bytes":6,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-f2cb7ec2edf3721f86331b75: start","job_status":"start","queue_duration_s":0.797622,"scheduling_latency_s":0.797643}2064vm-test-run-nixbot-gitlab> gitlab # [ 309.867570] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:35.170Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"f2cb7ec2edf3721f86331b75","created_at":"2026-07-01T08:10:34.310Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.321Z","job_size_bytes":6,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-f2cb7ec2edf3721f86331b75: done: 0.051777 sec","job_status":"done","queue_duration_s":0.797622,"scheduling_latency_s":0.797643,"redis_calls":10,"redis_duration_s":0.002809,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00062,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001651,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000538,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032561,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00057936,"duration_s":0.051777,"completed_at":"2026-07-01T08:10:35.170Z","load_balancing_strategy":"primary","db_duration_s":0.005193,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2065vm-test-run-nixbot-gitlab> gitlab # [ 309.912201] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:35.173Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"46f1cd6df6690e7a4a420393","created_at":"2026-07-01T08:10:34.408Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.412Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-46f1cd6df6690e7a4a420393: start","job_status":"start","queue_duration_s":0.760788,"scheduling_latency_s":0.760827}2066vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b/statuses2067vm-test-run-nixbot-gitlab> gitlab # [ 309.928033] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:35.211Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"46f1cd6df6690e7a4a420393","created_at":"2026-07-01T08:10:34.408Z","trace_propagation_headers":{"sentry-trace":"8b814a64dc8b4fc38c2fe89afa7fff47-045c36ff77fe4269","baggage":"sentry-trace_id=8b814a64dc8b4fc38c2fe89afa7fff47,sentry-release=Unknown"},"correlation_id":"01KWEBNM4M1NNVCVX8212GA0M1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:34.412Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-46f1cd6df6690e7a4a420393: done: 0.038111 sec","job_status":"done","queue_duration_s":0.760788,"scheduling_latency_s":0.760827,"redis_calls":8,"redis_duration_s":0.003338,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002696,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000642,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038111,"completed_at":"2026-07-01T08:10:35.211Z","load_balancing_strategy":"primary","db_duration_s":0.003394,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2068vm-test-run-nixbot-gitlab> gitlab # [ 310.116156] gitaly[1368]: time="2026-07-01T08:10:35.986Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNQ08MN07RY8AV2N4TX0Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:05.283" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:35.984" grpc.time_ms=1.254 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2069vm-test-run-nixbot-gitlab> gitlab # [ 310.157013] redis[1274]: 10 changes in 300 seconds. Saving...2070vm-test-run-nixbot-gitlab> gitlab # [ 310.160109] redis[1274]: Background saving started by pid 42272071vm-test-run-nixbot-gitlab> gitlab # [ 310.163836] redis[4227]: BGSAVE done, 609 keys saved, 0 keys skipped, 140987 bytes written.2072vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b/statuses, in 0.20 seconds)2073vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '70946e52cb9c0a8a279a04d1c7834a4d85e18e8b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-01T08:10:33.255Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2074vm-test-run-nixbot-gitlab> gitlab # [ 310.168128] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:36 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1182075vm-test-run-nixbot-gitlab> gitlab # [ 310.176425] redis[4227]: DB saved on disk2076vm-test-run-nixbot-gitlab> gitlab # [ 310.178621] redis[4227]: Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB2077vm-test-run-nixbot-gitlab> gitlab # [ 310.241356] redis[1274]: Background saving terminated with success2078vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b/statuses2079vm-test-run-nixbot-gitlab> gitlab # [ 311.313993] gitaly[1368]: time="2026-07-01T08:10:37.183Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNR50564A6AWWFGRAAYS8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:07.180" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:37.181" grpc.time_ms=1.293 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2080vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b/statuses, in 0.20 seconds)2081vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '70946e52cb9c0a8a279a04d1c7834a4d85e18e8b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-01T08:10:33.255Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2082vm-test-run-nixbot-gitlab> gitlab # [ 311.358932] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:37 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1392083vm-test-run-nixbot-gitlab> gitlab # [ 311.402481] postgres[1343]: [1343] LOG: checkpoint starting: time2084vm-test-run-nixbot-gitlab> gitlab # [ 312.021553] gitaly[1368]: time="2026-07-01T08:10:37.891Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNRTPBJM9P6V750FX5Q9V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:07.889" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:37.889" grpc.time_ms=1.281 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2085vm-test-run-nixbot-gitlab> gitlab # [ 312.110207] gitaly[1368]: time="2026-07-01T08:10:37.980Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNRTPBJM9P6V750FX5Q9V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T08:10:47.131" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:37.961" grpc.time_ms=18.202 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2086vm-test-run-nixbot-gitlab> gitlab # [ 312.192673] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.060Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"e16f496e4cb1ecd0ac931c99","created_at":"2026-07-01T08:10:38.049Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-fa66c1af129e43a7","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.057Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e16f496e4cb1ecd0ac931c99: start","job_status":"start","queue_duration_s":0.002995,"scheduling_latency_s":0.003065}2087vm-test-run-nixbot-gitlab> gitlab # [ 312.236806] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 3082088vm-test-run-nixbot-gitlab> gitlab # [ 312.242936] nixbot[3802]: {"ts": "2026-07-01T08:10:38.093Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b \"HTTP/1.1 201 Created\""}2089vm-test-run-nixbot-gitlab> gitlab # [ 312.320427] gitaly[1368]: time="2026-07-01T08:10:38.189Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNS4K999ZEAJ89M45KYTA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:08.187" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:38.188" grpc.time_ms=1.194 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2090vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b/statuses2091vm-test-run-nixbot-gitlab> gitlab # [ 312.397265] gitaly[1368]: time="2026-07-01T08:10:38.267Z" level=info msg="burden monitor stats" pid=1368 top_10_by_cpu="[]" top_10_by_memory="[]"2092vm-test-run-nixbot-gitlab> gitlab # [ 312.415127] gitaly[1368]: time="2026-07-01T08:10:38.285Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNS4K999ZEAJ89M45KYTA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T08:10:48.196" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:38.267" grpc.time_ms=16.728 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2093vm-test-run-nixbot-gitlab> gitlab # [ 312.446583] gitaly[1368]: time="2026-07-01T08:10:38.307Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBNRTPBJM9P6V750FX5Q9V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:38.307" grpc.time_ms=0.546 pid=1368 span.kind=server system=grpc2094vm-test-run-nixbot-gitlab> gitlab # [ 312.480943] gitaly[1368]: time="2026-07-01T08:10:38.351Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBNRTPBJM9P6V750FX5Q9V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:38.345" grpc.time_ms=4.964 pid=1368 span.kind=server system=grpc2095vm-test-run-nixbot-gitlab> gitlab # [ 312.508867] gitaly[1368]: time="2026-07-01T08:10:38.379Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNSAESXDFMR7GPFRG0YVE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:08.377" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:38.377" grpc.time_ms=1.056 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2096vm-test-run-nixbot-gitlab> gitlab # [ 312.531828] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2822097vm-test-run-nixbot-gitlab> gitlab # [ 312.536462] nixbot[3802]: {"ts": "2026-07-01T08:10:38.384Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b \"HTTP/1.1 201 Created\""}2098vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b/statuses, in 0.19 seconds)2099vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '70946e52cb9c0a8a279a04d1c7834a4d85e18e8b', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-01T08:10:33.255Z', 'started_at': None, 'finished_at': '2026-07-01T08:10:38.027Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '70946e52cb9c0a8a279a04d1c7834a4d85e18e8b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-01T08:10:38.338Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2100vm-test-run-nixbot-gitlab> gitlab # [ 312.545824] gitaly[1368]: time="2026-07-01T08:10:38.416Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWEBNRTPBJM9P6V750FX5Q9V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T08:10:38.415" grpc.time_ms=0.385 pid=1368 span.kind=server system=grpc2101vm-test-run-nixbot-gitlab> gitlab # [ 312.562055] gitaly[1368]: time="2026-07-01T08:10:38.427Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNRTPBJM9P6V750FX5Q9V diskcache=14917531-e18c-43c6-85da-247f6108767a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-01T08:11:08.249" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:38.249" pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2102vm-test-run-nixbot-gitlab> gitlab # [ 312.577150] gitaly[1368]: time="2026-07-01T08:10:38.427Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=131 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7488 command.oublock=0 command.real_time_ms=167 command.spawn_token_fork_ms=1 command.system_time_ms=62 command.user_time_ms=69 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNRTPBJM9P6V750FX5Q9V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-01T08:11:08.249" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:38.249" grpc.time_ms=177.529 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2103vm-test-run-nixbot-gitlab> gitlab # [ 312.596034] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 1332104vm-test-run-nixbot-gitlab> gitlab # [ 312.599528] gitaly[1368]: time="2026-07-01T08:10:38.463Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNSDX8ZXKK47WWEKWZ536 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:08.460" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:38.460" grpc.time_ms=2.325 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2105vm-test-run-nixbot-gitlab> gitlab # [ 312.616125] gitaly[1368]: time="2026-07-01T08:10:38.486Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNRTPBJM9P6V750FX5Q9V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-01T08:11:08.485" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:38.485" grpc.time_ms=0.805 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2106vm-test-run-nixbot-gitlab> gitlab # [ 312.645505] gitaly[1368]: time="2026-07-01T08:10:38.515Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNSDX8ZXKK47WWEKWZ536 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T08:10:48.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T08:10:38.504" grpc.time_ms=11.179 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2107vm-test-run-nixbot-gitlab> gitlab # [ 312.665067] gitaly[1368]: time="2026-07-01T08:10:38.519Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNRTPBJM9P6V750FX5Q9V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:08.518" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:38.518" grpc.time_ms=0.559 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2108vm-test-run-nixbot-gitlab> gitlab # [ 312.701247] gitaly[1368]: time="2026-07-01T08:10:38.571Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNRTPBJM9P6V750FX5Q9V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T08:11:08.570" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T08:10:38.570" grpc.time_ms=0.564 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2109vm-test-run-nixbot-gitlab> gitlab # [ 312.717146] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.583Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"2d36ed619aecf24feff07afe","created_at":"2026-07-01T08:10:38.581Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"e16f496e4cb1ecd0ac931c99","job_size_bytes":3,"pid":2429,"job_status":"deduplicated","message":"PipelineProcessWorker JID-2d36ed619aecf24feff07afe: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2110vm-test-run-nixbot-gitlab> gitlab # [ 312.734561] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.585Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"e16f496e4cb1ecd0ac931c99","created_at":"2026-07-01T08:10:38.049Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-fa66c1af129e43a7","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.057Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2429,"job_status":"rescheduled","message":"PipelineProcessWorker JID-e16f496e4cb1ecd0ac931c99: rescheduled"}2111vm-test-run-nixbot-gitlab> gitlab # [ 312.751293] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.593Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"e16f496e4cb1ecd0ac931c99","created_at":"2026-07-01T08:10:38.049Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-fa66c1af129e43a7","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.057Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e16f496e4cb1ecd0ac931c99: done: 0.532758 sec","job_status":"done","queue_duration_s":0.002995,"scheduling_latency_s":0.003065,"gitaly_calls":4,"gitaly_duration_s":0.189511,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.024656,"redis_read_bytes":262,"redis_write_bytes":23384,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002266,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002413,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000877,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.004067,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14465,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.008735,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000597,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001295,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.00214,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.06,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.016,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.021,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.029,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.271699,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.47118732,"duration_s":0.532758,"completed_at":"2026-07-01T08:10:38.593Z","load_balancing_strategy":"primary","db_duration_s":0.063449,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2112vm-test-run-nixbot-gitlab> gitlab # [ 312.805782] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.594Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"bb990722087c4e5ca3ff65b7","created_at":"2026-07-01T08:10:38.231Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.236Z","job_size_bytes":23,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-bb990722087c4e5ca3ff65b7: start","job_status":"start","queue_duration_s":0.358425,"scheduling_latency_s":0.358448}2113vm-test-run-nixbot-gitlab> gitlab # [ 312.822671] nixbot[3802]: {"ts": "2026-07-01T08:10:38.596Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b \"HTTP/1.1 201 Created\""}2114vm-test-run-nixbot-gitlab> gitlab # [ 312.825510] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1982115vm-test-run-nixbot-gitlab> gitlab # [ 312.828050] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.642Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"bb990722087c4e5ca3ff65b7","created_at":"2026-07-01T08:10:38.231Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.236Z","job_size_bytes":23,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-bb990722087c4e5ca3ff65b7: done: 0.047915 sec","job_status":"done","queue_duration_s":0.358425,"scheduling_latency_s":0.358448,"redis_calls":7,"redis_duration_s":0.0017959999999999999,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001354,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034195,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047915,"completed_at":"2026-07-01T08:10:38.642Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006153,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2116vm-test-run-nixbot-gitlab> gitlab # [ 312.871797] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.644Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"59ac8b8e6b8f7eb2e8038d46","created_at":"2026-07-01T08:10:38.435Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.438Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-59ac8b8e6b8f7eb2e8038d46: start","job_status":"start","queue_duration_s":0.205581,"scheduling_latency_s":0.205603}2117vm-test-run-nixbot-gitlab> gitlab # [ 312.888953] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.669Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"59ac8b8e6b8f7eb2e8038d46","created_at":"2026-07-01T08:10:38.435Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.438Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-59ac8b8e6b8f7eb2e8038d46: done: 0.0259 sec","job_status":"done","queue_duration_s":0.205581,"scheduling_latency_s":0.205603,"redis_calls":7,"redis_duration_s":0.002035,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00159,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000445,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0259,"completed_at":"2026-07-01T08:10:38.669Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00392,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2118vm-test-run-nixbot-gitlab> gitlab # [ 312.933361] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.671Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"2d9241493f4051eb6be59a24","created_at":"2026-07-01T08:10:38.448Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.451Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-2d9241493f4051eb6be59a24: start","job_status":"start","queue_duration_s":0.2195,"scheduling_latency_s":0.219519}2119vm-test-run-nixbot-gitlab> gitlab # [ 312.949605] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.699Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"2d9241493f4051eb6be59a24","created_at":"2026-07-01T08:10:38.448Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.451Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-2d9241493f4051eb6be59a24: done: 0.028046 sec","job_status":"done","queue_duration_s":0.2195,"scheduling_latency_s":0.219519,"redis_calls":7,"redis_duration_s":0.001542,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001126,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000416,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018069,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028046,"completed_at":"2026-07-01T08:10:38.699Z","load_balancing_strategy":"primary","db_duration_s":0.005509,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2120vm-test-run-nixbot-gitlab> gitlab # [ 312.992497] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.701Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"e3a39ff5cc84a926eae9995a","created_at":"2026-07-01T08:10:38.460Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.462Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-e3a39ff5cc84a926eae9995a: start","job_status":"start","queue_duration_s":0.238137,"scheduling_latency_s":0.23816}2121vm-test-run-nixbot-gitlab> gitlab # [ 313.008441] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.729Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"e3a39ff5cc84a926eae9995a","created_at":"2026-07-01T08:10:38.460Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.462Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-e3a39ff5cc84a926eae9995a: done: 0.028344 sec","job_status":"done","queue_duration_s":0.238137,"scheduling_latency_s":0.23816,"redis_calls":10,"redis_duration_s":0.002904,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001611,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000758,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000535,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028344,"completed_at":"2026-07-01T08:10:38.729Z","load_balancing_strategy":"primary","db_duration_s":0.004006,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2122vm-test-run-nixbot-gitlab> gitlab # [ 313.053058] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.730Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"d60ce5f7a5b99e1054839975","created_at":"2026-07-01T08:10:38.463Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.465Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-d60ce5f7a5b99e1054839975: start","job_status":"start","queue_duration_s":0.265025,"scheduling_latency_s":0.265042}2123vm-test-run-nixbot-gitlab> gitlab # [ 313.070142] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.759Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"d60ce5f7a5b99e1054839975","created_at":"2026-07-01T08:10:38.463Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.465Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-d60ce5f7a5b99e1054839975: done: 0.028876 sec","job_status":"done","queue_duration_s":0.265025,"scheduling_latency_s":0.265042,"redis_calls":7,"redis_duration_s":0.002349,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001843,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000506,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015372,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028876,"completed_at":"2026-07-01T08:10:38.759Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008142,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2124vm-test-run-nixbot-gitlab> gitlab # [ 313.114751] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.761Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"c67188a3f02ea775c446e3df","created_at":"2026-07-01T08:10:38.466Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.470Z","job_size_bytes":19,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-c67188a3f02ea775c446e3df: start","job_status":"start","queue_duration_s":0.290976,"scheduling_latency_s":0.291005}2125vm-test-run-nixbot-gitlab> gitlab # [ 313.131415] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.792Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"c67188a3f02ea775c446e3df","created_at":"2026-07-01T08:10:38.466Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.470Z","job_size_bytes":19,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-c67188a3f02ea775c446e3df: done: 0.03051 sec","job_status":"done","queue_duration_s":0.290976,"scheduling_latency_s":0.291005,"redis_calls":8,"redis_duration_s":0.002435,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001835,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0006,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017991,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03051,"completed_at":"2026-07-01T08:10:38.792Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006181,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2126vm-test-run-nixbot-gitlab> gitlab # [ 313.174709] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.793Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"272646da77a926b9675ac8ce","created_at":"2026-07-01T08:10:38.471Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.475Z","job_size_bytes":66,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-272646da77a926b9675ac8ce: start","job_status":"start","queue_duration_s":0.318436,"scheduling_latency_s":0.318452}2127vm-test-run-nixbot-gitlab> gitlab # [ 313.192945] nix-daemon[4090]: accepted connection from pid 4280, user nixbot2128vm-test-run-nixbot-gitlab> gitlab # [ 313.193955] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 862129vm-test-run-nixbot-gitlab> gitlab # [ 313.196089] gitlab-workhorse[1300]: time="2026-07-01T08:10:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672130vm-test-run-nixbot-gitlab> gitlab # [ 313.197966] gitlab-workhorse[1300]: gitlab 2001:db8:1::1 - - [2026/07/01:08:10:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 682131vm-test-run-nixbot-gitlab> gitlab # [ 313.199937] gitaly[1368]: time="2026-07-01T08:10:38.984Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWEBNSXJBQYNYA2G0Q5F0NWW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:38.975" grpc.time_ms=9.441 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc2132vm-test-run-nixbot-gitlab> gitlab # [ 313.213644] gitaly[1368]: time="2026-07-01T08:10:39.055Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNT0AVHGSRKKPN33SNT37 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:39.045" pid=1368 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc2133vm-test-run-nixbot-gitlab> gitlab # [ 313.224418] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.825Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"272646da77a926b9675ac8ce","created_at":"2026-07-01T08:10:38.471Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.475Z","job_size_bytes":66,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-272646da77a926b9675ac8ce: done: 0.031477 sec","job_status":"done","queue_duration_s":0.318436,"scheduling_latency_s":0.318452,"redis_calls":7,"redis_duration_s":0.011588000000000001,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005313,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.006275,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031477,"completed_at":"2026-07-01T08:10:38.824Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003002,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2134vm-test-run-nixbot-gitlab> gitlab # [ 313.268863] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.833Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"6aa4412f4179f099960ec6f3","created_at":"2026-07-01T08:10:38.476Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.478Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-6aa4412f4179f099960ec6f3: start","job_status":"start","queue_duration_s":0.355412,"scheduling_latency_s":0.355433}2135vm-test-run-nixbot-gitlab> gitlab # [ 313.285084] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.881Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"6aa4412f4179f099960ec6f3","created_at":"2026-07-01T08:10:38.476Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.478Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-6aa4412f4179f099960ec6f3: done: 0.047472 sec","job_status":"done","queue_duration_s":0.355412,"scheduling_latency_s":0.355433,"redis_calls":7,"redis_duration_s":0.0077859999999999995,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004867,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002919,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047472,"completed_at":"2026-07-01T08:10:38.881Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006175,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2136vm-test-run-nixbot-gitlab> gitlab # [ 313.330206] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.883Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"9fc0d4c8bf6a135f64f85154","created_at":"2026-07-01T08:10:38.528Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.530Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-9fc0d4c8bf6a135f64f85154: start","job_status":"start","queue_duration_s":0.353178,"scheduling_latency_s":0.353211}2137vm-test-run-nixbot-gitlab> gitlab # [ 313.347035] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.923Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"9fc0d4c8bf6a135f64f85154","created_at":"2026-07-01T08:10:38.528Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.530Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-9fc0d4c8bf6a135f64f85154: done: 0.040162 sec","job_status":"done","queue_duration_s":0.353178,"scheduling_latency_s":0.353211,"redis_calls":7,"redis_duration_s":0.0025329999999999997,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002028,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000505,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040162,"completed_at":"2026-07-01T08:10:38.923Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00652,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2138vm-test-run-nixbot-gitlab> gitlab # [ 313.390533] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.926Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"82c9240bf7cd5ce735bda6c9","created_at":"2026-07-01T08:10:38.531Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.534Z","job_size_bytes":6,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-82c9240bf7cd5ce735bda6c9: start","job_status":"start","queue_duration_s":0.392087,"scheduling_latency_s":0.392119}2139vm-test-run-nixbot-gitlab> gitlab # [ 313.406811] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.949Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"82c9240bf7cd5ce735bda6c9","created_at":"2026-07-01T08:10:38.531Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.534Z","job_size_bytes":6,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-82c9240bf7cd5ce735bda6c9: done: 0.023258 sec","job_status":"done","queue_duration_s":0.392087,"scheduling_latency_s":0.392119,"redis_calls":7,"redis_duration_s":0.001735,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001327,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000408,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014612,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023258,"completed_at":"2026-07-01T08:10:38.949Z","load_balancing_strategy":"primary","db_duration_s":0.001528,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2140vm-test-run-nixbot-gitlab> gitlab # [ 313.449492] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.951Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"8b21a478675b7ac268c17f86","created_at":"2026-07-01T08:10:38.534Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.539Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-8b21a478675b7ac268c17f86: start","job_status":"start","queue_duration_s":0.410995,"scheduling_latency_s":0.41102}2141vm-test-run-nixbot-gitlab> gitlab # [ 313.465076] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.997Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"8b21a478675b7ac268c17f86","created_at":"2026-07-01T08:10:38.534Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.539Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-8b21a478675b7ac268c17f86: done: 0.046192 sec","job_status":"done","queue_duration_s":0.410995,"scheduling_latency_s":0.41102,"redis_calls":10,"redis_duration_s":0.0028859999999999997,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000677,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001584,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000625,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00048594,"duration_s":0.046192,"completed_at":"2026-07-01T08:10:38.997Z","load_balancing_strategy":"primary","db_duration_s":0.006929,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2142vm-test-run-nixbot-gitlab> gitlab # [ 313.509772] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:38.999Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"387b775915a9475180404716","created_at":"2026-07-01T08:10:38.585Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.587Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-387b775915a9475180404716: start","job_status":"start","queue_duration_s":0.412016,"scheduling_latency_s":0.412039}2143vm-test-run-nixbot-gitlab> gitlab # [ 313.526162] gitaly[1368]: time="2026-07-01T08:10:39.055Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWEBNT0AVHGSRKKPN33SNT37 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T08:10:39.045" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1368 remote_ip="2001:db8:1::1" span.kind=server system=grpc2144vm-test-run-nixbot-gitlab> gitlab # [ 313.539772] nix-daemon[4090]: accepted connection from pid 4336, user nixbot2145vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b/statuses2146vm-test-run-nixbot-gitlab> gitlab # [ 313.540844] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:39.267Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"387b775915a9475180404716","created_at":"2026-07-01T08:10:38.585Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-1c7173dbdb014d82","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:38.587Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-387b775915a9475180404716: done: 0.268246 sec","job_status":"done","queue_duration_s":0.412016,"scheduling_latency_s":0.412039,"gitaly_calls":3,"gitaly_duration_s":0.090107,"redis_calls":65,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.018203,"redis_read_bytes":144,"redis_write_bytes":16874,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.003724,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001814,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":20,"redis_queues_duration_s":0.002049,"redis_queues_read_bytes":21,"redis_queues_write_bytes":11223,"redis_queues_metadata_calls":18,"redis_queues_metadata_duration_s":0.003487,"redis_queues_metadata_read_bytes":28,"redis_queues_metadata_write_bytes":2886,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00028,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000362,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002763,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":26,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":26,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.013,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.013,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.136127,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.23565326,"duration_s":0.268246,"completed_at":"2026-07-01T08:10:39.267Z","load_balancing_strategy":"primary","db_duration_s":0.03437,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2147vm-test-run-nixbot-gitlab> gitlab # [ 313.589826] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:39.269Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"59ad4961a8610a5cc06e13ab","created_at":"2026-07-01T08:10:39.079Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-14274562c5d94675","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:39.081Z","job_size_bytes":23,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-59ad4961a8610a5cc06e13ab: start","job_status":"start","queue_duration_s":0.187853,"scheduling_latency_s":0.187875}2148vm-test-run-nixbot-gitlab> gitlab # [ 313.606870] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:39.304Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"59ad4961a8610a5cc06e13ab","created_at":"2026-07-01T08:10:39.079Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-14274562c5d94675","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:39.081Z","job_size_bytes":23,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-59ad4961a8610a5cc06e13ab: done: 0.035402 sec","job_status":"done","queue_duration_s":0.187853,"scheduling_latency_s":0.187875,"redis_calls":7,"redis_duration_s":0.001397,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001042,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000355,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0261,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035402,"completed_at":"2026-07-01T08:10:39.304Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005318,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2149vm-test-run-nixbot-gitlab> gitlab # [ 313.650274] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:39.306Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"77be8cb7fe4043b2701183bf","created_at":"2026-07-01T08:10:39.174Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-14274562c5d94675","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:39.177Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-77be8cb7fe4043b2701183bf: start","job_status":"start","queue_duration_s":0.128703,"scheduling_latency_s":0.128723}2150vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/70946e52cb9c0a8a279a04d1c7834a4d85e18e8b/statuses, in 0.16 seconds)2151vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '70946e52cb9c0a8a279a04d1c7834a4d85e18e8b', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-01T08:10:33.255Z', 'started_at': None, 'finished_at': '2026-07-01T08:10:38.027Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '70946e52cb9c0a8a279a04d1c7834a4d85e18e8b', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-07-01T08:10:38.338Z', 'started_at': None, 'finished_at': '2026-07-01T08:10:38.543Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2152vm-test-run-nixbot-gitlab> gitlab # [ 313.667491] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:39.328Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"77be8cb7fe4043b2701183bf","created_at":"2026-07-01T08:10:39.174Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-14274562c5d94675","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:39.177Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-77be8cb7fe4043b2701183bf: done: 0.022186 sec","job_status":"done","queue_duration_s":0.128703,"scheduling_latency_s":0.128723,"redis_calls":7,"redis_duration_s":0.0012929999999999999,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000932,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000361,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022186,"completed_at":"2026-07-01T08:10:39.328Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003769,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2153vm-test-run-nixbot-gitlab> (finished: subtest: eval and build post commit statuses, in 6.08 seconds)2154vm-test-run-nixbot-gitlab> (finished: run the VM test script, in 379.42 seconds)2155vm-test-run-nixbot-gitlab> gitlab # [ 313.712473] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:39.329Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"1cc89dc18d7f8fea81aaf670","created_at":"2026-07-01T08:10:39.186Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-14274562c5d94675","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:39.188Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-1cc89dc18d7f8fea81aaf670: start","job_status":"start","queue_duration_s":0.141221,"scheduling_latency_s":0.141238}2156vm-test-run-nixbot-gitlab> test script finished in 379.46s2157vm-test-run-nixbot-gitlab> cleanup2158vm-test-run-nixbot-gitlab> kill QemuMachine (pid 13)2159vm-test-run-nixbot-gitlab> gitlab # [ 313.727833] sidekiq-cluster[2429]: {"severity":"INFO","time":"2026-07-01T08:10:39.361Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"1cc89dc18d7f8fea81aaf670","created_at":"2026-07-01T08:10:39.186Z","trace_propagation_headers":{"sentry-trace":"33cb3cc0745148c3b82d5eddaabd76bb-14274562c5d94675","baggage":"sentry-trace_id=33cb3cc0745148c3b82d5eddaabd76bb,sentry-release=Unknown"},"correlation_id":"01KWEBNRTPBJM9P6V750FX5Q9V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-01T08:10:39.188Z","job_size_bytes":3,"pid":2429,"sidekiq_tid":"ma01","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-1cc89dc18d7f8fea81aaf670: done: 0.031976 sec","job_status":"done","queue_duration_s":0.141221,"scheduling_latency_s":0.141238,"redis_calls":7,"redis_duration_s":0.0015609999999999999,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00112,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000441,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_countqemu-system-aarch64: terminating on signal 15 from pid 6 (/nix/store/lqn6mbgzzdrqq2qkwddcmxj9z6amdd86-python3-3.13.13/bin/python3.13)2160vm-test-run-nixbot-gitlab> gitlab # "2161vm-test-run-nixbot-gitlab> (finished: cleanup, in 1.79 seconds)21622163post-build step Upload coverage to codecov: ok2164Skipping codecov: project=Mic92/nixbot attr=aarch64-linux.nixbot-gitlab