these 13 derivations will be built: /nix/store/z6drqa73xidh9fd5cyr2w7p267mia5ji-python3.14-nixbot.drv /nix/store/cb28yxwx7kby8ny5cmhahinjzmgzr3lw-unit-nixbot.service.drv /nix/store/9pzylm9d6p1z3ggdky60knan60awgk5n-system-units.drv /nix/store/53b7yjcvlpimhklv69l3y5k085yizpkz-etc.drv /nix/store/246a115vxnla1vji75x2d47yg7wls8dk-activate.drv /nix/store/vdxnqfysdd4mjp5l91cb2f3i673liax4-nixos-system-gitlab-test.drv /nix/store/gj0ksbszg3a8m492c5g09x37jzfbnbgk-closure-info.drv /nix/store/x37yby403mr5lwf4yrmyiyy3bgvqwmfb-closure-info.drv /nix/store/kalvigpy1pxms1wmxmcx6gdv41mrs54i-run-nixos-vm.drv /nix/store/i2cs88zqlr12adqkfngm62wci5d6jsms-nixos-vm.drv /nix/store/kh8i5adap98hl0qb4ah7n962g5hwrvx6-driverConfiguration.json.drv /nix/store/g8a74q93vv18gnpln8jzxw5pbsiqm07c-nixos-test-driver-nixbot-gitlab.drv /nix/store/m3lrlcshn6z4z4r6nq7ii7h7xlziwp4c-vm-test-run-nixbot-gitlab.drv Failed to find a machine for remote build! derivation: z6drqa73xidh9fd5cyr2w7p267mia5ji-python3.14-nixbot.drv required (system, features): (aarch64-linux, []) 1 available machines: (systems, maxjobs, supportedFeatures, mandatoryFeatures) ([aarch64-darwin, x86_64-darwin], 8, [big-parallel, recursive-nix], []) building '/nix/store/z6drqa73xidh9fd5cyr2w7p267mia5ji-python3.14-nixbot.drv' python3.14-nixbot> tribuchet: building on eliza python3.14-nixbot> Sourcing python-remove-tests-dir-hook python3.14-nixbot> Sourcing python-catch-conflicts-hook.sh python3.14-nixbot> Sourcing python-remove-bin-bytecode-hook.sh python3.14-nixbot> Sourcing pypa-build-hook python3.14-nixbot> Using pypaBuildPhase python3.14-nixbot> Sourcing python-runtime-deps-check-hook python3.14-nixbot> Using pythonRuntimeDepsCheckHook python3.14-nixbot> Sourcing pypa-install-hook python3.14-nixbot> Using pypaInstallPhase python3.14-nixbot> Sourcing python-imports-check-hook.sh python3.14-nixbot> Using pythonImportsCheckPhase python3.14-nixbot> Sourcing python-namespaces-hook python3.14-nixbot> Sourcing python-catch-conflicts-hook.sh python3.14-nixbot> Running phase: unpackPhase python3.14-nixbot> unpacking source archive /nix/store/21vxlka8ywrq9b7kfs1l64jg4fgp8851-nixbot python3.14-nixbot> source root is nixbot python3.14-nixbot> setting SOURCE_DATE_EPOCH to timestamp 315619200 of file "nixbot/pyproject.toml" python3.14-nixbot> Running phase: patchPhase python3.14-nixbot> Running phase: updateAutotoolsGnuConfigScriptsPhase python3.14-nixbot> Running phase: configurePhase python3.14-nixbot> no configure script, doing nothing python3.14-nixbot> Running phase: buildPhase python3.14-nixbot> Executing pypaBuildPhase python3.14-nixbot> Creating a wheel... python3.14-nixbot> pypa build flags: --no-isolation --outdir dist/ --wheel python3.14-nixbot> * Getting build dependencies for wheel... python3.14-nixbot> * Building wheel... python3.14-nixbot> Successfully built nixbot-0.0.1-py3-none-any.whl python3.14-nixbot> Finished creating a wheel... python3.14-nixbot> Finished executing pypaBuildPhase python3.14-nixbot> Running phase: pythonRuntimeDepsCheckHook python3.14-nixbot> Executing pythonRuntimeDepsCheck python3.14-nixbot> Checking runtime dependencies for nixbot-0.0.1-py3-none-any.whl python3.14-nixbot> Finished executing pythonRuntimeDepsCheck python3.14-nixbot> Running phase: installPhase python3.14-nixbot> Executing pypaInstallPhase python3.14-nixbot> Successfully installed nixbot-0.0.1-py3-none-any.whl python3.14-nixbot> Finished executing pypaInstallPhase python3.14-nixbot> Running phase: pythonOutputDistPhase python3.14-nixbot> Executing pythonOutputDistPhase python3.14-nixbot> Finished executing pythonOutputDistPhase python3.14-nixbot> Running phase: fixupPhase python3.14-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/wnink6vk1aadwracxisj9mwy5dkq4w2c-python3.14-nixbot python3.14-nixbot> checking for references to /build/ in /nix/store/wnink6vk1aadwracxisj9mwy5dkq4w2c-python3.14-nixbot... python3.14-nixbot> patching script interpreter paths in /nix/store/wnink6vk1aadwracxisj9mwy5dkq4w2c-python3.14-nixbot python3.14-nixbot> stripping (with command strip and flags -S -p) in /nix/store/wnink6vk1aadwracxisj9mwy5dkq4w2c-python3.14-nixbot/lib /nix/store/wnink6vk1aadwracxisj9mwy5dkq4w2c-python3.14-nixbot/bin python3.14-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/d58yysy2jvmljm43l4ywff9jqb4g8b1y-python3.14-nixbot-dist python3.14-nixbot> checking for references to /build/ in /nix/store/d58yysy2jvmljm43l4ywff9jqb4g8b1y-python3.14-nixbot-dist... python3.14-nixbot> patching script interpreter paths in /nix/store/d58yysy2jvmljm43l4ywff9jqb4g8b1y-python3.14-nixbot-dist python3.14-nixbot> Rewriting #!/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14 to #!/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6 python3.14-nixbot> wrapping `/nix/store/wnink6vk1aadwracxisj9mwy5dkq4w2c-python3.14-nixbot/bin/nixbot'... python3.14-nixbot> Executing pythonRemoveTestsDir python3.14-nixbot> Finished executing pythonRemoveTestsDir python3.14-nixbot> Running phase: pythonCatchConflictsPhase python3.14-nixbot> Running phase: pythonRemoveBinBytecodePhase python3.14-nixbot> Running phase: pythonImportsCheckPhase python3.14-nixbot> Executing pythonImportsCheckPhase building '/nix/store/cb28yxwx7kby8ny5cmhahinjzmgzr3lw-unit-nixbot.service.drv' unit-nixbot.service> tribuchet: building on eliza unit-nixbot.service> structuredAttrs is enabled building '/nix/store/9pzylm9d6p1z3ggdky60knan60awgk5n-system-units.drv' system-units> tribuchet: building on eliza building '/nix/store/53b7yjcvlpimhklv69l3y5k085yizpkz-etc.drv' etc> tribuchet: building on eliza building '/nix/store/246a115vxnla1vji75x2d47yg7wls8dk-activate.drv' activate> tribuchet: building on eliza building '/nix/store/vdxnqfysdd4mjp5l91cb2f3i673liax4-nixos-system-gitlab-test.drv' nixos-system-gitlab-test> tribuchet: building on eliza nixos-system-gitlab-test> structuredAttrs is enabled building '/nix/store/gj0ksbszg3a8m492c5g09x37jzfbnbgk-closure-info.drv' closure-info> tribuchet: building on eliza closure-info> structuredAttrs is enabled building '/nix/store/x37yby403mr5lwf4yrmyiyy3bgvqwmfb-closure-info.drv' closure-info> tribuchet: building on eliza closure-info> structuredAttrs is enabled building '/nix/store/kalvigpy1pxms1wmxmcx6gdv41mrs54i-run-nixos-vm.drv' run-nixos-vm> tribuchet: building on eliza building '/nix/store/i2cs88zqlr12adqkfngm62wci5d6jsms-nixos-vm.drv' nixos-vm> tribuchet: building on eliza building '/nix/store/kh8i5adap98hl0qb4ah7n962g5hwrvx6-driverConfiguration.json.drv' driverConfiguration.json> tribuchet: building on eliza driverConfiguration.json> structuredAttrs is enabled building '/nix/store/g8a74q93vv18gnpln8jzxw5pbsiqm07c-nixos-test-driver-nixbot-gitlab.drv' nixos-test-driver-nixbot-gitlab> tribuchet: building on eliza nixos-test-driver-nixbot-gitlab> Running type check (enable/disable: config.skipTypeCheck) nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck nixos-test-driver-nixbot-gitlab> All checks passed! nixos-test-driver-nixbot-gitlab> Linting test script (enable/disable: config.skipLint) nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint nixos-test-driver-nixbot-gitlab> All checks passed! Failed to find a machine for remote build! derivation: m3lrlcshn6z4z4r6nq7ii7h7xlziwp4c-vm-test-run-nixbot-gitlab.drv required (system, features): (aarch64-linux, [kvm, nixos-test]) 1 available machines: (systems, maxjobs, supportedFeatures, mandatoryFeatures) ([aarch64-darwin, x86_64-darwin], 8, [big-parallel, recursive-nix], []) building '/nix/store/m3lrlcshn6z4z4r6nq7ii7h7xlziwp4c-vm-test-run-nixbot-gitlab.drv' vm-test-run-nixbot-gitlab> tribuchet: building on eliza vm-test-run-nixbot-gitlab> Machine state will be reset. To keep it, pass --keep-machine-state vm-test-run-nixbot-gitlab> start all VLans vm-test-run-nixbot-gitlab> (finished: start all VLans, in 0.00 seconds) vm-test-run-nixbot-gitlab> Test will time out and terminate in 3600 seconds vm-test-run-nixbot-gitlab> run the VM test script vm-test-run-nixbot-gitlab> start all VMs vm-test-run-nixbot-gitlab> gitlab: starting vm vm-test-run-nixbot-gitlab> gitlab # Disk image does not exist, creating the virtualisation disk image... vm-test-run-nixbot-gitlab> gitlab: QEMU running (pid 46) vm-test-run-nixbot-gitlab> gitlab # Formatting '/build/vm-state-gitlab/tmp.x1TUVf1NLi', fmt=raw size=1073741824 vm-test-run-nixbot-gitlab> gitlab # mke2fs 1.47.4 (6-Mar-2025) vm-test-run-nixbot-gitlab> gitlab # Discarding device blocks: 0/262144 done vm-test-run-nixbot-gitlab> gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes vm-test-run-nixbot-gitlab> gitlab # Filesystem UUID: b9118612-c3ed-4b3b-9aec-5cdb286e46a5 vm-test-run-nixbot-gitlab> (finished: start all VMs, in 66.50 seconds) vm-test-run-nixbot-gitlab> gitlab # Superblock backups stored on blocks: vm-test-run-nixbot-gitlab> gitlab: waiting for unit setup-gitlab.service vm-test-run-nixbot-gitlab> gitlab # 32768, 98304, 163840, 229376 vm-test-run-nixbot-gitlab> gitlab: waiting for the VM to finish booting vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # Allocating group tables: 0/8 done vm-test-run-nixbot-gitlab> gitlab # Writing inode tables: 0/8 done vm-test-run-nixbot-gitlab> gitlab # Creating journal (8192 blocks): done vm-test-run-nixbot-gitlab> gitlab # Writing superblocks and filesystem accounting information: 0/8 done vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # Virtualisation disk image created. vm-test-run-nixbot-gitlab> gitlab # Creating Nix store image... vm-test-run-nixbot-gitlab> gitlab # Created Nix store image. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 2026 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] KASLR enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] random: crng init done vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Machine model: linux,dummy-virt vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] efi: UEFI not found. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Zone ranges: vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMA32 empty vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Device empty vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Movable zone start for each node vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Early memory node ranges vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: probing for conduit method from DT. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: Trusted OS migration not required vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: SMC Calling Convention v1.1 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Detected PIPT I-cache on CPU0 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: Spectre-v4 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: Spectre-BHB vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] alternatives: applying boot alternatives vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/vpks99861m3nr1g0l38zd7qfhh34rs5c-nixos-system-gitlab-test/init regInfo=/nix/store/2zw1046a25833w43jalc6wr0w88a6b13-closure-info/registration console=ttyAMA0,115200n8 console=tty0 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/2zw1046a25833w43jalc6wr0w88a6b13-closure-info/registration", will be passed to user space. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] software IO TLB: area num 4. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Fallback order for Node 0: 0 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1572864 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Policy zone: Normal vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] allocated 12582912 bytes of page_ext vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: Hierarchical RCU implementation. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: RCU event tracing is enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Rude variant of Tasks RCU enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: 256 SPIs implemented vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f30000 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns vm-test-run-nixbot-gitlab> gitlab # [ 0.000030] arm-pv: using stolen time PV vm-test-run-nixbot-gitlab> gitlab # [ 0.000406] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) vm-test-run-nixbot-gitlab> gitlab # [ 0.000618] Console: colour dummy device 80x25 vm-test-run-nixbot-gitlab> gitlab # [ 0.000625] printk: legacy console [tty0] enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.000810] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) vm-test-run-nixbot-gitlab> gitlab # [ 0.000817] pid_max: default: 32768 minimum: 301 vm-test-run-nixbot-gitlab> gitlab # [ 0.000904] LSM: initializing lsm=capability,landlock,yama,bpf,ima vm-test-run-nixbot-gitlab> gitlab # [ 0.001039] landlock: Up and running. vm-test-run-nixbot-gitlab> gitlab # [ 0.001042] Yama: becoming mindful. vm-test-run-nixbot-gitlab> gitlab # [ 0.001475] LSM support for eBPF active vm-test-run-nixbot-gitlab> gitlab # [ 0.001719] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.001836] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.003040] cacheinfo: Unable to detect cache hierarchy for CPU 0 vm-test-run-nixbot-gitlab> gitlab # [ 0.003729] rcu: Hierarchical SRCU implementation. vm-test-run-nixbot-gitlab> gitlab # [ 0.003733] rcu: Max phase no-delay instances is 1000. vm-test-run-nixbot-gitlab> gitlab # [ 0.003891] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level vm-test-run-nixbot-gitlab> gitlab # [ 0.004988] fsl-mc MSI: its@8080000 domain created vm-test-run-nixbot-gitlab> gitlab # [ 0.005082] EFI services will not be available. vm-test-run-nixbot-gitlab> gitlab # [ 0.005258] smp: Bringing up secondary CPUs ... vm-test-run-nixbot-gitlab> gitlab # [ 0.005802] Detected PIPT I-cache on CPU1 vm-test-run-nixbot-gitlab> gitlab # [ 0.005867] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000 vm-test-run-nixbot-gitlab> gitlab # [ 0.005939] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000 vm-test-run-nixbot-gitlab> gitlab # [ 0.006011] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40] vm-test-run-nixbot-gitlab> gitlab # [ 0.006847] Detected PIPT I-cache on CPU2 vm-test-run-nixbot-gitlab> gitlab # [ 0.006912] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000 vm-test-run-nixbot-gitlab> gitlab # [ 0.006984] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000 vm-test-run-nixbot-gitlab> gitlab # [ 0.007056] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40] vm-test-run-nixbot-gitlab> gitlab # [ 0.007853] Detected PIPT I-cache on CPU3 vm-test-run-nixbot-gitlab> gitlab # [ 0.007923] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000 vm-test-run-nixbot-gitlab> gitlab # [ 0.007994] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000 vm-test-run-nixbot-gitlab> gitlab # [ 0.008067] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40] vm-test-run-nixbot-gitlab> gitlab # [ 0.008415] smp: Brought up 1 node, 4 CPUs vm-test-run-nixbot-gitlab> gitlab # [ 0.008442] SMP: Total of 4 processors activated. vm-test-run-nixbot-gitlab> gitlab # [ 0.008444] CPU: All CPU(s) started at EL1 vm-test-run-nixbot-gitlab> gitlab # [ 0.008453] CPU features: detected: Branch Target Identification vm-test-run-nixbot-gitlab> gitlab # [ 0.008457] CPU features: detected: ARMv8.4 Translation Table Level vm-test-run-nixbot-gitlab> gitlab # [ 0.008459] CPU features: detected: Instruction cache invalidation not required for I/D coherence vm-test-run-nixbot-gitlab> gitlab # [ 0.008463] CPU features: detected: Data cache clean to the PoU not required for I/D coherence vm-test-run-nixbot-gitlab> gitlab # [ 0.008466] CPU features: detected: Common not Private translations vm-test-run-nixbot-gitlab> gitlab # [ 0.008469] CPU features: detected: CRC32 instructions vm-test-run-nixbot-gitlab> gitlab # [ 0.008471] CPU features: detected: Data cache clean to Point of Deep Persistence vm-test-run-nixbot-gitlab> gitlab # [ 0.008474] CPU features: detected: Data cache clean to Point of Persistence vm-test-run-nixbot-gitlab> gitlab # [ 0.008477] CPU features: detected: Data independent timing control (DIT) vm-test-run-nixbot-gitlab> gitlab # [ 0.008480] CPU features: detected: E0PD vm-test-run-nixbot-gitlab> gitlab # [ 0.008482] CPU features: detected: Enhanced Counter Virtualization vm-test-run-nixbot-gitlab> gitlab # [ 0.008485] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) vm-test-run-nixbot-gitlab> gitlab # [ 0.008487] CPU features: detected: Enhanced Virtualization Traps vm-test-run-nixbot-gitlab> gitlab # [ 0.008490] CPU features: detected: Fine Grained Traps vm-test-run-nixbot-gitlab> gitlab # [ 0.008493] CPU features: detected: Generic authentication (architected QARMA5 algorithm) vm-test-run-nixbot-gitlab> gitlab # [ 0.008497] CPU features: detected: RCpc load-acquire (LDAPR) vm-test-run-nixbot-gitlab> gitlab # [ 0.008499] CPU features: detected: LSE atomic instructions vm-test-run-nixbot-gitlab> gitlab # [ 0.008501] CPU features: detected: Privileged Access Never vm-test-run-nixbot-gitlab> gitlab # [ 0.008504] CPU features: detected: PMUv3 vm-test-run-nixbot-gitlab> gitlab # [ 0.008506] CPU features: detected: RAS Extension Support vm-test-run-nixbot-gitlab> gitlab # [ 0.008508] CPU features: detected: RASv1p1 Extension Support vm-test-run-nixbot-gitlab> gitlab # [ 0.008511] CPU features: detected: Random Number Generator vm-test-run-nixbot-gitlab> gitlab # [ 0.008513] CPU features: detected: Speculation barrier (SB) vm-test-run-nixbot-gitlab> gitlab # [ 0.008515] CPU features: detected: Stage-2 Force Write-Back vm-test-run-nixbot-gitlab> gitlab # [ 0.008518] CPU features: detected: TLB range maintenance instructions vm-test-run-nixbot-gitlab> gitlab # [ 0.008521] CPU features: detected: Speculative Store Bypassing Safe (SSBS) vm-test-run-nixbot-gitlab> gitlab # [ 0.008606] alternatives: applying system-wide alternatives vm-test-run-nixbot-gitlab> gitlab # [ 0.011487] CPU features: detected: BBM Level 2 without TLB conflict abort vm-test-run-nixbot-gitlab> gitlab # [ 0.011701] Memory: 5963808K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved) vm-test-run-nixbot-gitlab> gitlab # [ 0.013327] devtmpfs: initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.016397] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.016469] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). vm-test-run-nixbot-gitlab> gitlab # [ 0.016695] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL vm-test-run-nixbot-gitlab> gitlab # [ 0.016700] 0 pages in range for non-PLT usage vm-test-run-nixbot-gitlab> gitlab # [ 0.016702] 508288 pages in range for PLT usage vm-test-run-nixbot-gitlab> gitlab # [ 0.016835] pinctrl core: initialized pinctrl subsystem vm-test-run-nixbot-gitlab> gitlab # [ 0.017649] DMI not present or invalid. vm-test-run-nixbot-gitlab> gitlab # [ 0.021368] NET: Registered PF_NETLINK/PF_ROUTE protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.023755] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations vm-test-run-nixbot-gitlab> gitlab # [ 0.024010] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations vm-test-run-nixbot-gitlab> gitlab # [ 0.025978] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations vm-test-run-nixbot-gitlab> gitlab # [ 0.025999] audit: initializing netlink subsys (disabled) vm-test-run-nixbot-gitlab> gitlab # [ 0.026221] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1 vm-test-run-nixbot-gitlab> gitlab # [ 0.026680] thermal_sys: Registered thermal governor 'fair_share' vm-test-run-nixbot-gitlab> gitlab # [ 0.026682] thermal_sys: Registered thermal governor 'bang_bang' vm-test-run-nixbot-gitlab> gitlab # [ 0.026686] thermal_sys: Registered thermal governor 'step_wise' vm-test-run-nixbot-gitlab> gitlab # [ 0.026689] thermal_sys: Registered thermal governor 'user_space' vm-test-run-nixbot-gitlab> gitlab # [ 0.026691] thermal_sys: Registered thermal governor 'power_allocator' vm-test-run-nixbot-gitlab> gitlab # [ 0.026747] cpuidle: using governor ladder vm-test-run-nixbot-gitlab> gitlab # [ 0.026767] cpuidle: using governor menu vm-test-run-nixbot-gitlab> gitlab # [ 0.026961] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. vm-test-run-nixbot-gitlab> gitlab # [ 0.027089] ASID allocator initialised with 65536 entries vm-test-run-nixbot-gitlab> gitlab # [ 0.028290] Serial: AMBA PL011 UART driver vm-test-run-nixbot-gitlab> gitlab # [ 0.033486] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 vm-test-run-nixbot-gitlab> gitlab # [ 0.033759] printk: console [ttyAMA0] enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.039730] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.039740] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page vm-test-run-nixbot-gitlab> gitlab # [ 0.039745] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.039750] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page vm-test-run-nixbot-gitlab> gitlab # [ 0.039754] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.039758] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page vm-test-run-nixbot-gitlab> gitlab # [ 0.039763] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.039767] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page vm-test-run-nixbot-gitlab> gitlab # [ 0.046931] fbcon: Taking over console vm-test-run-nixbot-gitlab> gitlab # [ 0.046950] ACPI: Interpreter disabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.048506] iommu: Default domain type: Translated vm-test-run-nixbot-gitlab> gitlab # [ 0.048516] iommu: DMA domain TLB invalidation policy: strict mode vm-test-run-nixbot-gitlab> gitlab # [ 0.061056] SCSI subsystem initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.061622] usbcore: registered new interface driver usbfs vm-test-run-nixbot-gitlab> gitlab # [ 0.061672] usbcore: registered new interface driver hub vm-test-run-nixbot-gitlab> gitlab # [ 0.061694] usbcore: registered new device driver usb vm-test-run-nixbot-gitlab> gitlab # [ 0.062166] pps_core: LinuxPPS API ver. 1 registered vm-test-run-nixbot-gitlab> gitlab # [ 0.062172] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti vm-test-run-nixbot-gitlab> gitlab # [ 0.062184] PTP clock support registered vm-test-run-nixbot-gitlab> gitlab # [ 0.062256] EDAC MC: Ver: 3.0.0 vm-test-run-nixbot-gitlab> gitlab # [ 0.062660] scmi_core: SCMI protocol bus registered vm-test-run-nixbot-gitlab> gitlab # [ 0.063234] FPGA manager framework vm-test-run-nixbot-gitlab> gitlab # [ 0.064609] vgaarb: loaded vm-test-run-nixbot-gitlab> gitlab # [ 0.065213] clocksource: Switched to clocksource arch_sys_counter vm-test-run-nixbot-gitlab> gitlab # [ 0.067427] VFS: Disk quotas dquot_6.6.0 vm-test-run-nixbot-gitlab> gitlab # [ 0.067460] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) vm-test-run-nixbot-gitlab> gitlab # [ 0.067927] netfs: FS-Cache loaded vm-test-run-nixbot-gitlab> gitlab # [ 0.068046] pnp: PnP ACPI: disabled vm-test-run-nixbot-gitlab> gitlab # [ 0.073452] NET: Registered PF_INET protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.074982] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.116080] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.116145] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.116203] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.116612] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.117445] TCP: Hash tables configured (established 65536 bind 65536) vm-test-run-nixbot-gitlab> gitlab # [ 0.117579] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.117658] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.117817] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.118043] NET: Registered PF_UNIX/PF_LOCAL protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.118169] NET: Registered PF_XDP protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.118186] PCI: CLS 0 bytes, default 64 vm-test-run-nixbot-gitlab> gitlab # [ 0.118386] Trying to unpack rootfs image as initramfs... vm-test-run-nixbot-gitlab> gitlab # [ 0.125703] kvm [1]: HYP mode not available vm-test-run-nixbot-gitlab> gitlab # [ 0.183051] Initialise system trusted keyrings vm-test-run-nixbot-gitlab> gitlab # [ 0.189440] workingset: timestamp_bits=42 max_order=21 bucket_order=0 vm-test-run-nixbot-gitlab> gitlab # [ 0.190238] squashfs: version 4.0 (2009/01/31) Phillip Lougher vm-test-run-nixbot-gitlab> gitlab # [ 0.190332] 9p: Installing v9fs 9p2000 file system support vm-test-run-nixbot-gitlab> gitlab # [ 0.209387] Key type asymmetric registered vm-test-run-nixbot-gitlab> gitlab # [ 0.209393] Asymmetric key parser 'x509' registered vm-test-run-nixbot-gitlab> gitlab # [ 0.209521] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) vm-test-run-nixbot-gitlab> gitlab # [ 0.209727] io scheduler mq-deadline registered vm-test-run-nixbot-gitlab> gitlab # [ 0.209738] io scheduler kyber registered vm-test-run-nixbot-gitlab> gitlab # [ 0.219583] pl061_gpio 9030000.pl061: PL061 GPIO chip registered vm-test-run-nixbot-gitlab> gitlab # [ 0.221561] ledtrig-cpu: registered to indicate activity on CPUs vm-test-run-nixbot-gitlab> gitlab # [ 0.222149] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: vm-test-run-nixbot-gitlab> gitlab # [ 0.222170] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.222184] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.222193] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.222219] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits vm-test-run-nixbot-gitlab> gitlab # [ 0.222246] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] vm-test-run-nixbot-gitlab> gitlab # [ 0.222350] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 vm-test-run-nixbot-gitlab> gitlab # [ 0.222372] pci_bus 0000:00: root bus resource [bus 00-ff] vm-test-run-nixbot-gitlab> gitlab # [ 0.222379] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.222384] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.222388] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.222484] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.223265] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.223607] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] vm-test-run-nixbot-gitlab> gitlab # [ 0.223632] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.223674] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.223699] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.224544] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.224877] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] vm-test-run-nixbot-gitlab> gitlab # [ 0.224896] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.224944] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.225805] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.226126] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f] vm-test-run-nixbot-gitlab> gitlab # [ 0.226144] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.226192] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.227010] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.227337] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] vm-test-run-nixbot-gitlab> gitlab # [ 0.227355] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.227396] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.228234] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.228566] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f] vm-test-run-nixbot-gitlab> gitlab # [ 0.228584] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.228626] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.229450] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.229782] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f] vm-test-run-nixbot-gitlab> gitlab # [ 0.229800] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.229842] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.230673] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.230992] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f] vm-test-run-nixbot-gitlab> gitlab # [ 0.231010] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.231057] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.231081] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.231896] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.232222] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.232263] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.233102] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.233447] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.233490] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.234191] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.234504] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.234920] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.235322] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] vm-test-run-nixbot-gitlab> gitlab # [ 0.235346] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.235393] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.236190] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.236523] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] vm-test-run-nixbot-gitlab> gitlab # [ 0.236542] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.236590] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.237704] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.237714] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.237719] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.237789] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.237873] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.237943] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238018] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238091] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238165] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238241] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238316] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238398] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238499] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238574] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238608] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238636] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238664] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238699] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238732] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238767] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238801] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238836] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238871] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238901] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238930] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238958] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.238992] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.239019] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.239047] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.239074] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.239101] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.239136] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.239170] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.239205] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.239245] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.239253] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.239258] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.241556] pci 0000:00:0a.0: enabling device (0000 -> 0002) vm-test-run-nixbot-gitlab> gitlab # [ 0.259480] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.262602] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.265756] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.269165] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.273486] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.276715] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.279615] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.282664] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) vm-test-run-nixbot-gitlab> gitlab # [ 0.287016] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) vm-test-run-nixbot-gitlab> gitlab # [ 0.289644] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.293026] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.299851] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.302122] msm_serial: driver initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.302327] SuperH (H)SCI(F) driver initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.302393] STM32 USART driver initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.331031] loop: module loaded vm-test-run-nixbot-gitlab> gitlab # [ 0.331265] virtio_blk virtio4: 4/0/0 default/read/poll queues vm-test-run-nixbot-gitlab> gitlab # [ 0.333719] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) vm-test-run-nixbot-gitlab> gitlab # [ 0.336130] virtio_blk virtio5: 4/0/0 default/read/poll queues vm-test-run-nixbot-gitlab> gitlab # [ 0.338901] virtio_blk virtio5: [vdb] 12394776 512-byte logical blocks (6.35 GB/5.91 GiB) vm-test-run-nixbot-gitlab> gitlab # [ 0.341819] megasas: 07.734.00.00-rc1 vm-test-run-nixbot-gitlab> gitlab # [ 0.342727] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.346238] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.346258] Intel/Sharp Extended Query Table at 0x0031 vm-test-run-nixbot-gitlab> gitlab # [ 0.348344] Using buffer write method vm-test-run-nixbot-gitlab> gitlab # [ 0.348380] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.350679] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.350697] Intel/Sharp Extended Query Table at 0x0031 vm-test-run-nixbot-gitlab> gitlab # [ 0.352896] Using buffer write method vm-test-run-nixbot-gitlab> gitlab # [ 0.352913] Concatenating MTD devices: vm-test-run-nixbot-gitlab> gitlab # [ 0.352916] (0): "0.flash" vm-test-run-nixbot-gitlab> gitlab # [ 0.352919] (1): "0.flash" vm-test-run-nixbot-gitlab> gitlab # [ 0.352922] into device "0.flash" vm-test-run-nixbot-gitlab> gitlab # [ 0.581420] Freeing initrd memory: 27184K vm-test-run-nixbot-gitlab> gitlab # [ 0.590767] tun: Universal TUN/TAP device driver, 1.6 vm-test-run-nixbot-gitlab> gitlab # [ 0.595621] thunder_xcv, ver 1.0 vm-test-run-nixbot-gitlab> gitlab # [ 0.595695] thunder_bgx, ver 1.0 vm-test-run-nixbot-gitlab> gitlab # [ 0.595726] nicpf, ver 1.0 vm-test-run-nixbot-gitlab> gitlab # [ 0.596661] e1000: Intel(R) PRO/1000 Network Driver vm-test-run-nixbot-gitlab> gitlab # [ 0.596667] e1000: Copyright (c) 1999-2006 Intel Corporation. vm-test-run-nixbot-gitlab> gitlab # [ 0.596703] e1000e: Intel(R) PRO/1000 Network Driver vm-test-run-nixbot-gitlab> gitlab # [ 0.596711] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. vm-test-run-nixbot-gitlab> gitlab # [ 0.596772] igb: Intel(R) Gigabit Ethernet Network Driver vm-test-run-nixbot-gitlab> gitlab # [ 0.596777] igb: Copyright (c) 2007-2014 Intel Corporation. vm-test-run-nixbot-gitlab> gitlab # [ 0.596811] igbvf: Intel(R) Gigabit Virtual Function Network Driver vm-test-run-nixbot-gitlab> gitlab # [ 0.596816] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. vm-test-run-nixbot-gitlab> gitlab # [ 0.597059] sky2: driver version 1.30 vm-test-run-nixbot-gitlab> gitlab # [ 0.599601] usbcore: registered new interface driver usb-storage vm-test-run-nixbot-gitlab> gitlab # [ 0.599708] usbcore: registered new interface driver usbserial_generic vm-test-run-nixbot-gitlab> gitlab # [ 0.599730] usbserial: USB Serial support registered for generic vm-test-run-nixbot-gitlab> gitlab # [ 0.599977] ehci-pci 0000:00:0a.0: EHCI Host Controller vm-test-run-nixbot-gitlab> gitlab # [ 0.600045] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1 vm-test-run-nixbot-gitlab> gitlab # [ 0.600494] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000 vm-test-run-nixbot-gitlab> gitlab # [ 0.600674] hv_vmbus: registering driver hyperv_keyboard vm-test-run-nixbot-gitlab> gitlab # [ 0.602065] rtc-pl031 9010000.pl031: registered as rtc0 vm-test-run-nixbot-gitlab> gitlab # [ 0.602096] rtc-pl031 9010000.pl031: setting system clock to 2026-10-04T11:26:22 UTC (1791113182) vm-test-run-nixbot-gitlab> gitlab # [ 0.602566] i2c_dev: i2c /dev entries driver vm-test-run-nixbot-gitlab> gitlab # [ 0.606983] sdhci: Secure Digital Host Controller Interface driver vm-test-run-nixbot-gitlab> gitlab # [ 0.606991] sdhci: Copyright(c) Pierre Ossman vm-test-run-nixbot-gitlab> gitlab # [ 0.607402] Synopsys Designware Multimedia Card Interface Driver vm-test-run-nixbot-gitlab> gitlab # [ 0.607989] sdhci-pltfm: SDHCI platform and OF driver helper vm-test-run-nixbot-gitlab> gitlab # [ 0.610472] hid: raw HID events driver (C) Jiri Kosina vm-test-run-nixbot-gitlab> gitlab # [ 0.610923] usbcore: registered new interface driver usbhid vm-test-run-nixbot-gitlab> gitlab # [ 0.610929] usbhid: USB HID core driver vm-test-run-nixbot-gitlab> gitlab # [ 0.613289] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00 vm-test-run-nixbot-gitlab> gitlab # [ 0.613894] hub 1-0:1.0: USB hub found vm-test-run-nixbot-gitlab> gitlab # [ 0.613967] hub 1-0:1.0: 6 ports detected vm-test-run-nixbot-gitlab> gitlab # [ 0.644877] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available vm-test-run-nixbot-gitlab> gitlab # [ 0.647334] drop_monitor: Initializing network drop monitor service vm-test-run-nixbot-gitlab> gitlab # [ 0.647560] NET: Registered PF_INET6 protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.648894] Segment Routing with IPv6 vm-test-run-nixbot-gitlab> gitlab # [ 0.648927] In-situ OAM (IOAM) with IPv6 vm-test-run-nixbot-gitlab> gitlab # [ 0.648996] NET: Registered PF_PACKET protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.649176] 9pnet: Installing 9P2000 support vm-test-run-nixbot-gitlab> gitlab # [ 0.651075] Key type dns_resolver registered vm-test-run-nixbot-gitlab> gitlab # [ 0.657600] registered taskstats version 1 vm-test-run-nixbot-gitlab> gitlab # [ 0.657843] Loading compiled-in X.509 certificates vm-test-run-nixbot-gitlab> gitlab # [ 0.666101] Demotion targets for Node 0: null vm-test-run-nixbot-gitlab> gitlab # [ 0.666239] Key type .fscrypt registered vm-test-run-nixbot-gitlab> gitlab # [ 0.666243] Key type fscrypt-provisioning registered vm-test-run-nixbot-gitlab> gitlab # [ 0.666374] ima: No TPM chip found, activating TPM-bypass! vm-test-run-nixbot-gitlab> gitlab # [ 0.666391] ima: Allocated hash algorithm: sha1 vm-test-run-nixbot-gitlab> gitlab # [ 0.666411] ima: No architecture policies found vm-test-run-nixbot-gitlab> gitlab # [ 0.667093] input: gpio-keys as /devices/platform/gpio-keys/input/input0 vm-test-run-nixbot-gitlab> gitlab # [ 0.686179] clk: Disabling unused clocks vm-test-run-nixbot-gitlab> gitlab # [ 0.686202] PM: genpd: Disabling unused power domains vm-test-run-nixbot-gitlab> gitlab # [ 0.689656] Freeing unused kernel memory: 4736K vm-test-run-nixbot-gitlab> gitlab # [ 0.689834] Run /init as init process vm-test-run-nixbot-gitlab> gitlab # [ 0.716574] systemd[1]: Successfully made /usr/ read-only. vm-test-run-nixbot-gitlab> gitlab # [ 0.865247] usb 1-1: new high-speed USB device number 2 using ehci-pci vm-test-run-nixbot-gitlab> gitlab # [ 1.017086] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 vm-test-run-nixbot-gitlab> gitlab # [ 1.050582] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) vm-test-run-nixbot-gitlab> gitlab # [ 1.050618] systemd[1]: Detected virtualization qemu. vm-test-run-nixbot-gitlab> gitlab # [ 1.050658] systemd[1]: Detected architecture arm64. vm-test-run-nixbot-gitlab> gitlab # [ 1.050666] systemd[1]: Running in initrd. vm-test-run-nixbot-gitlab> gitlab # [ 1.051100] systemd[1]: Initializing machine ID from random generator. vm-test-run-nixbot-gitlab> gitlab # [ 1.051236] systemd[1]: Hostname set to . vm-test-run-nixbot-gitlab> gitlab # [ 1.113676] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0 vm-test-run-nixbot-gitlab> gitlab # [ 1.227457] systemd[1]: bpf-restrict-fs: LSM BPF program attached vm-test-run-nixbot-gitlab> gitlab # [ 1.237255] usb 1-2: new high-speed USB device number 3 using ehci-pci vm-test-run-nixbot-gitlab> gitlab # [ 1.289560] systemd[1]: Queued start job for default target Initrd Default Target. vm-test-run-nixbot-gitlab> gitlab # [ 1.312136] systemd[1]: Created slice Slice /system/modprobe. vm-test-run-nixbot-gitlab> gitlab # [ 1.312397] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 1.312432] systemd[1]: Expecting device /dev/disk/by-label/nix-store... vm-test-run-nixbot-gitlab> gitlab # [ 1.312446] systemd[1]: Expecting device /dev/disk/by-label/nixos... vm-test-run-nixbot-gitlab> gitlab # [ 1.312471] systemd[1]: Reached target Path Units. vm-test-run-nixbot-gitlab> gitlab # [ 1.312487] systemd[1]: Reached target Slice Units. vm-test-run-nixbot-gitlab> gitlab # [ 1.312504] systemd[1]: Reached target Swaps. vm-test-run-nixbot-gitlab> gitlab # [ 1.312523] systemd[1]: Reached target Timer Units. vm-test-run-nixbot-gitlab> gitlab # [ 1.312695] systemd[1]: Listening on D-Bus System Message Bus Socket. vm-test-run-nixbot-gitlab> gitlab # [ 1.312864] systemd[1]: Listening on Journal Socket (/dev/log). vm-test-run-nixbot-gitlab> gitlab # [ 1.313019] systemd[1]: Listening on Journal Sockets. vm-test-run-nixbot-gitlab> gitlab # [ 1.313168] systemd[1]: Listening on udev Control Socket. vm-test-run-nixbot-gitlab> gitlab # [ 1.313402] systemd[1]: Listening on udev Kernel Socket. vm-test-run-nixbot-gitlab> gitlab # [ 1.313425] systemd[1]: Reached target Socket Units. vm-test-run-nixbot-gitlab> gitlab # [ 1.315106] systemd[1]: Starting Create List of Static Device Nodes... vm-test-run-nixbot-gitlab> gitlab # [ 1.316911] systemd[1]: Starting Load Kernel Module 9pnet_virtio... vm-test-run-nixbot-gitlab> gitlab # [ 1.316984] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs vm-test-run-nixbot-gitlab> gitlab # [ 1.318809] systemd[1]: Mounting Kernel Configuration File System... vm-test-run-nixbot-gitlab> gitlab # [ 1.334907] systemd[1]: Starting Journal Service... vm-test-run-nixbot-gitlab> gitlab # [ 1.337026] systemd[1]: Starting Load Kernel Modules... vm-test-run-nixbot-gitlab> gitlab # [ 1.337162] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os vm-test-run-nixbot-gitlab> gitlab # [ 1.339156] systemd[1]: Starting Coldplug All udev Devices... vm-test-run-nixbot-gitlab> gitlab # [ 1.343997] systemd[1]: Finished Create List of Static Device Nodes. vm-test-run-nixbot-gitlab> gitlab # [ 1.345682] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 1.346179] systemd[1]: Finished Load Kernel Module 9pnet_virtio. vm-test-run-nixbot-gitlab> gitlab # [ 1.346524] systemd[1]: Mounted Kernel Configuration File System. vm-test-run-nixbot-gitlab> gitlab # [ 1.353369] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... vm-test-run-nixbot-gitlab> gitlab # [ 1.365930] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0 vm-test-run-nixbot-gitlab> gitlab # [ 1.366209] [drm] features: -virgl +edid -resource_blob -host_visible vm-test-run-nixbot-gitlab> gitlab # [ 1.366214] [drm] features: -context_init vm-test-run-nixbot-gitlab> gitlab # [ 1.367524] [drm] number of scanouts: 1 vm-test-run-nixbot-gitlab> gitlab # [ 1.367542] [drm] number of cap sets: 0 vm-test-run-nixbot-gitlab> gitlab # [ 1.371475] systemd-journald[94]: Collecting audit messages is disabled. vm-test-run-nixbot-gitlab> gitlab # [ 1.371604] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic vm-test-run-nixbot-gitlab> gitlab # [ 1.371630] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0 vm-test-run-nixbot-gitlab> gitlab # [ 1.377693] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. vm-test-run-nixbot-gitlab> gitlab # [ 1.377991] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev vm-test-run-nixbot-gitlab> gitlab # [ 1.381405] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. vm-test-run-nixbot-gitlab> gitlab # [ 1.385736] Console: switching to colour frame buffer device 160x50 vm-test-run-nixbot-gitlab> gitlab # [ 1.389599] systemd[1]: Starting Create Static Device Nodes in /dev... vm-test-run-nixbot-gitlab> gitlab # [ 1.393734] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device vm-test-run-nixbot-gitlab> gitlab # [ 1.399337] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 vm-test-run-nixbot-gitlab> gitlab # [ 1.401334] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0 vm-test-run-nixbot-gitlab> gitlab # [ 1.407612] systemd[1]: Finished Create Static Device Nodes in /dev. vm-test-run-nixbot-gitlab> gitlab # [ 1.408020] systemd[1]: Reached target Preparation for Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 1.408316] systemd[1]: Reached target Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 1.413675] systemd[1]: Starting Rule-based Manager for Device Events and Files... vm-test-run-nixbot-gitlab> gitlab # [ 1.414572] systemd[1]: Finished Load Kernel Modules. vm-test-run-nixbot-gitlab> gitlab # [ 1.416807] systemd[1]: Starting Apply Kernel Variables... vm-test-run-nixbot-gitlab> gitlab # [ 1.442508] systemd-modules-load[95]: Using 4 probe threads vm-test-run-nixbot-gitlab> gitlab # [ 1.444838] systemd[1]: Started Journal Service. vm-test-run-nixbot-gitlab> gitlab # [ 1.445735] systemd-modules-load[95]: Module 'virtio_balloon' is built in vm-test-run-nixbot-gitlab> gitlab # [ 1.446890] systemd-modules-load[95]: Module 'virtio_console' is built in vm-test-run-nixbot-gitlab> gitlab # [ 1.449455] systemd-modules-load[95]: Module 'virtio_rng' is built in vm-test-run-nixbot-gitlab> gitlab # [ 1.450547] systemd-modules-load[95]: Inserted module 'dm_mod' vm-test-run-nixbot-gitlab> gitlab # [ 1.453481] systemd-modules-load[95]: Inserted module 'virtio_gpu' vm-test-run-nixbot-gitlab> gitlab # [ 1.454555] systemd[1]: Starting Create System Files and Directories... vm-test-run-nixbot-gitlab> gitlab # [ 1.457131] systemd[1]: Finished Apply Kernel Variables. vm-test-run-nixbot-gitlab> gitlab # [ 1.469806] systemd-udevd[104]: Using default interface naming scheme 'v261'. vm-test-run-nixbot-gitlab> gitlab # [ 1.472082] systemd[1]: Finished Create System Files and Directories. vm-test-run-nixbot-gitlab> gitlab # [ 1.490421] systemd[1]: Started Rule-based Manager for Device Events and Files. vm-test-run-nixbot-gitlab> gitlab # [ 1.516266] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 1.559533] systemd-vconsole-setup[129]: Configuration of first virtual console was skipped, ignoring remaining ones. vm-test-run-nixbot-gitlab> gitlab # [ 1.562511] systemd[1]: Finished Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 1.980398] systemd[1]: Finished Coldplug All udev Devices. vm-test-run-nixbot-gitlab> gitlab # [ 1.981747] systemd[1]: Reached target System Initialization. vm-test-run-nixbot-gitlab> gitlab # [ 1.983042] systemd[1]: Reached target Basic System. vm-test-run-nixbot-gitlab> gitlab # [ 2.103462] (udev-worker)[135]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. vm-test-run-nixbot-gitlab> gitlab # [ 2.108634] (udev-worker)[135]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 2.111424] (udev-worker)[124]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 2.118263] systemd[1]: Found device /dev/disk/by-label/nix-store. vm-test-run-nixbot-gitlab> gitlab # [ 2.132042] systemd[1]: Found device /dev/disk/by-label/nixos. vm-test-run-nixbot-gitlab> gitlab # [ 2.134432] systemd[1]: Reached target Initrd Root Device. vm-test-run-nixbot-gitlab> gitlab # [ 2.136621] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... vm-test-run-nixbot-gitlab> gitlab # [ 2.174202] systemd-fsck[151]: nixos: clean, 12/65536 files, 13019/262144 blocks vm-test-run-nixbot-gitlab> gitlab # [ 2.176941] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. vm-test-run-nixbot-gitlab> gitlab # [ 2.341349] systemd[1]: Mounting /sysroot... vm-test-run-nixbot-gitlab> gitlab # [ 2.372480] EXT4-fs (vda): mounted filesystem b9118612-c3ed-4b3b-9aec-5cdb286e46a5 r/w with ordered data mode. Quota mode: none. vm-test-run-nixbot-gitlab> gitlab # [ 2.374315] systemd[1]: Mounted /sysroot. vm-test-run-nixbot-gitlab> gitlab # [ 2.375854] systemd[1]: Reached target Initrd Root File System. vm-test-run-nixbot-gitlab> gitlab # [ 2.378311] systemd[1]: Starting Mountpoints Configured in the Real Root... vm-test-run-nixbot-gitlab> gitlab # [ 2.412732] systemd-sysroot-fstab-check[159]: /sysroot should be mounted in the initrd, will request daemon-reload. vm-test-run-nixbot-gitlab> gitlab # [ 2.418104] systemd[1]: Reload requested from client PID 159 ('systemd-sysroot') (unit initrd-parse-etc.service)... vm-test-run-nixbot-gitlab> gitlab # [ 2.421785] systemd[1]: Reloading... vm-test-run-nixbot-gitlab> gitlab # [ 2.510558] systemd[1]: Reloading finished in 94 ms. vm-test-run-nixbot-gitlab> gitlab # [ 2.565032] systemd-sysroot-fstab-check[159]: Requesting initrd-fs.target/start/replace... vm-test-run-nixbot-gitlab> gitlab # [ 2.567402] systemd-sysroot-fstab-check[159]: Requesting swap.target/start/replace... vm-test-run-nixbot-gitlab> gitlab # [ 2.571211] systemd[1]: Starting Load Kernel Module 9pnet_virtio... vm-test-run-nixbot-gitlab> gitlab # [ 2.572741] systemd[1]: initrd-parse-etc.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 2.574224] systemd[1]: Finished Mountpoints Configured in the Real Root. vm-test-run-nixbot-gitlab> gitlab # [ 2.576213] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. vm-test-run-nixbot-gitlab> gitlab # [ 2.596509] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 2.597646] systemd[1]: Finished Load Kernel Module 9pnet_virtio. vm-test-run-nixbot-gitlab> gitlab # [ 2.600322] (udev-worker)[142]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory vm-test-run-nixbot-gitlab> gitlab # [ 2.602938] (udev-worker)[142]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory vm-test-run-nixbot-gitlab> gitlab # [ 2.609060] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 2.610389] systemd[1]: Stopped Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 2.611371] systemd[1]: Stopping Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 2.613482] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 2.628987] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 2.630116] systemd[1]: Stopped Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 2.630897] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 2.650178] systemd-vconsole-setup[182]: Configuration of first virtual console was skipped, ignoring remaining ones. vm-test-run-nixbot-gitlab> gitlab # [ 2.652343] systemd[1]: Finished Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 3.344266] systemd[1]: Mounting /sysroot/nix/.ro-store... vm-test-run-nixbot-gitlab> gitlab # [ 3.346509] systemd[1]: Mounting /sysroot/nix/.rw-store... vm-test-run-nixbot-gitlab> gitlab # [ 3.348850] systemd[1]: Mounting /sysroot/run... vm-test-run-nixbot-gitlab> gitlab # [ 3.365057] systemd[1]: Mounting /sysroot/tmp/shared... vm-test-run-nixbot-gitlab> gitlab # [ 3.369763] systemd[1]: Mounting /sysroot/tmp/xchg... vm-test-run-nixbot-gitlab> gitlab # [ 3.378187] systemd[1]: Mounted /sysroot/nix/.rw-store. vm-test-run-nixbot-gitlab> gitlab # [ 3.383631] systemd[1]: Starting rw-sysroot-nix-store.service... vm-test-run-nixbot-gitlab> gitlab # [ 3.393250] systemd[1]: Mounted /sysroot/run. vm-test-run-nixbot-gitlab> gitlab # [ 3.397900] erofs (device vdb): mounted with root inode @ nid 128. vm-test-run-nixbot-gitlab> gitlab # [ 3.402820] systemd[1]: Mounted /sysroot/nix/.ro-store. vm-test-run-nixbot-gitlab> gitlab # [ 3.404314] systemd[1]: Mounted /sysroot/tmp/shared. vm-test-run-nixbot-gitlab> gitlab # [ 3.405541] systemd[1]: Mounted /sysroot/tmp/xchg. vm-test-run-nixbot-gitlab> gitlab # [ 3.406645] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.408358] systemd[1]: Finished rw-sysroot-nix-store.service. vm-test-run-nixbot-gitlab> gitlab # [ 4.342268] systemd[1]: Mounting /sysroot/nix/store... vm-test-run-nixbot-gitlab> gitlab # [ 4.390141] systemd[1]: Mounted /sysroot/nix/store. vm-test-run-nixbot-gitlab> gitlab # [ 4.391489] systemd[1]: Reached target Initrd File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 4.392876] systemd[1]: Starting Find NixOS closure... vm-test-run-nixbot-gitlab> gitlab # [ 4.394627] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... vm-test-run-nixbot-gitlab> gitlab # [ 4.433414] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. vm-test-run-nixbot-gitlab> gitlab # [ 4.435338] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.444833] systemd[1]: Finished Find NixOS closure. vm-test-run-nixbot-gitlab> gitlab # [ 4.446066] systemd[1]: Reached target Initrd Default Target. vm-test-run-nixbot-gitlab> gitlab # [ 4.447369] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... vm-test-run-nixbot-gitlab> gitlab # [ 4.467224] systemd[1]: Stopped target Initrd Default Target. vm-test-run-nixbot-gitlab> gitlab # [ 4.468648] systemd[1]: Stopped target Basic System. vm-test-run-nixbot-gitlab> gitlab # [ 4.469794] systemd[1]: Stopped target Initrd Root Device. vm-test-run-nixbot-gitlab> gitlab # [ 4.471049] systemd[1]: Stopped target Path Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.472269] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.473946] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 4.475710] systemd[1]: Stopped target Slice Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.476921] systemd[1]: Stopped target Socket Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.479852] systemd[1]: Stopped target System Initialization. vm-test-run-nixbot-gitlab> gitlab # [ 4.481397] systemd[1]: Stopped target Swaps. vm-test-run-nixbot-gitlab> gitlab # [ 4.482707] systemd[1]: Stopped target Timer Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.484209] systemd[1]: dbus.socket: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.485541] systemd[1]: Closed D-Bus System Message Bus Socket. vm-test-run-nixbot-gitlab> gitlab # [ 4.488094] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.490763] systemd[1]: Stopped Find NixOS closure. vm-test-run-nixbot-gitlab> gitlab # [ 4.491998] systemd[1]: Starting Load Kernel Module 9pnet_virtio... vm-test-run-nixbot-gitlab> gitlab # [ 4.493443] systemd[1]: Starting rw-sysroot-nix-store.service... vm-test-run-nixbot-gitlab> gitlab # [ 4.494816] systemd[1]: systemd-sysctl.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.496451] systemd[1]: Stopped Apply Kernel Variables. vm-test-run-nixbot-gitlab> gitlab # [ 4.497654] systemd[1]: systemd-modules-load.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.499263] systemd[1]: Stopped Load Kernel Modules. vm-test-run-nixbot-gitlab> gitlab # [ 4.500528] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.502288] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. vm-test-run-nixbot-gitlab> gitlab # [ 4.504041] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.505658] systemd[1]: Stopped Create System Files and Directories. vm-test-run-nixbot-gitlab> gitlab # [ 4.507125] systemd[1]: Stopped target Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 4.508495] systemd[1]: Stopped target Preparation for Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 4.510080] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.511742] systemd[1]: Stopped Coldplug All udev Devices. vm-test-run-nixbot-gitlab> gitlab # [ 4.513059] systemd[1]: Stopping Rule-based Manager for Device Events and Files... vm-test-run-nixbot-gitlab> gitlab # [ 4.514687] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.516330] systemd[1]: Stopped Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 4.517456] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.518976] systemd[1]: Finished rw-sysroot-nix-store.service. vm-test-run-nixbot-gitlab> gitlab # [ 4.520230] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.521703] systemd[1]: Finished Load Kernel Module 9pnet_virtio. vm-test-run-nixbot-gitlab> gitlab # [ 4.522949] systemd[1]: initrd-cleanup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.524319] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. vm-test-run-nixbot-gitlab> gitlab # [ 4.525596] systemd[1]: systemd-udevd.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.526885] systemd[1]: Stopped Rule-based Manager for Device Events and Files. vm-test-run-nixbot-gitlab> gitlab # [ 4.528315] systemd[1]: systemd-udevd.service: Consumed 1.934s CPU time over 3.096s wall clock time, 31.1M memory peak. vm-test-run-nixbot-gitlab> gitlab # [ 4.530222] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.531586] systemd[1]: Closed udev Control Socket. vm-test-run-nixbot-gitlab> gitlab # [ 4.532691] systemd[1]: Starting Cleanup udev Database... vm-test-run-nixbot-gitlab> gitlab # [ 4.533706] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.535080] systemd[1]: Stopped Create Static Device Nodes in /dev. vm-test-run-nixbot-gitlab> gitlab # [ 4.536199] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.537596] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. vm-test-run-nixbot-gitlab> gitlab # [ 4.538821] systemd[1]: kmod-static-nodes.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.540015] systemd[1]: Stopped Create List of Static Device Nodes. vm-test-run-nixbot-gitlab> gitlab # [ 4.580849] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.582605] systemd[1]: Finished Cleanup udev Database. vm-test-run-nixbot-gitlab> gitlab # [ 4.583876] systemd[1]: Reached target Switch Root. vm-test-run-nixbot-gitlab> gitlab # [ 4.585107] systemd[1]: Starting NixOS Activation... vm-test-run-nixbot-gitlab> gitlab # [ 4.689470] initrd-nixos-activation-start[221]: booting system configuration /nix/store/vpks99861m3nr1g0l38zd7qfhh34rs5c-nixos-system-gitlab-test vm-test-run-nixbot-gitlab> gitlab # [ 4.722594] initrd-nixos-activation-start[221]: running activation script... vm-test-run-nixbot-gitlab> gitlab # [ 4.935405] initrd-nixos-activation-start[244]: setting up /etc... vm-test-run-nixbot-gitlab> gitlab # [ 5.036295] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 5.038128] systemd[1]: Finished NixOS Activation. vm-test-run-nixbot-gitlab> gitlab # [ 5.039337] systemd[1]: Starting Switch Root... vm-test-run-nixbot-gitlab> gitlab # [ 5.078750] systemd[1]: Switching root. vm-test-run-nixbot-gitlab> gitlab # [ 5.219427] systemd-journald[94]: Received SIGTERM from PID 1 (systemd). vm-test-run-nixbot-gitlab> gitlab # [ 5.772850] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) vm-test-run-nixbot-gitlab> gitlab # [ 5.772928] systemd[1]: Detected virtualization qemu. vm-test-run-nixbot-gitlab> gitlab # [ 5.772976] systemd[1]: Detected architecture arm64. vm-test-run-nixbot-gitlab> gitlab # [ 5.773072] systemd[1]: Detected first boot. vm-test-run-nixbot-gitlab> gitlab # [ 5.773820] systemd[1]: Initializing machine ID from random generator. vm-test-run-nixbot-gitlab> gitlab # [ 5.971192] systemd[1]: bpf-restrict-fs: LSM BPF program attached vm-test-run-nixbot-gitlab> gitlab # [ 6.053413] systemd[1]: Applying preset policy. vm-test-run-nixbot-gitlab> gitlab # [ 6.219816] systemd[1]: Populated /etc with preset unit settings. vm-test-run-nixbot-gitlab> gitlab # [ 6.383484] systemd[1]: initrd-switch-root.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 6.384271] systemd[1]: Stopped initrd-switch-root.service. vm-test-run-nixbot-gitlab> gitlab # [ 6.386284] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. vm-test-run-nixbot-gitlab> gitlab # [ 6.390208] systemd[1]: Created slice Slice /system/getty. vm-test-run-nixbot-gitlab> gitlab # [ 6.392252] systemd[1]: Created slice GitLab DevOps Platform Slice. vm-test-run-nixbot-gitlab> gitlab # [ 6.394583] systemd[1]: Created slice User and Session Slice. vm-test-run-nixbot-gitlab> gitlab # [ 6.395132] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 6.395748] systemd[1]: Started Forward Password Requests to Wall Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 6.396341] systemd[1]: Expecting device /dev/hvc0... vm-test-run-nixbot-gitlab> gitlab # [ 6.396954] systemd[1]: Expecting device /dev/ttyAMA0... vm-test-run-nixbot-gitlab> gitlab # [ 6.401371] systemd[1]: Reached target Local Encrypted Volumes. vm-test-run-nixbot-gitlab> gitlab # [ 6.401918] systemd[1]: Stopped target initrd-fs.target. vm-test-run-nixbot-gitlab> gitlab # [ 6.402322] systemd[1]: Stopped target initrd-root-fs.target. vm-test-run-nixbot-gitlab> gitlab # [ 6.402696] systemd[1]: Stopped target initrd-switch-root.target. vm-test-run-nixbot-gitlab> gitlab # [ 6.403087] systemd[1]: Reached target Virtual Machines and Containers. vm-test-run-nixbot-gitlab> gitlab # [ 6.403496] systemd[1]: Reached target Path Units. vm-test-run-nixbot-gitlab> gitlab # [ 6.403883] systemd[1]: Reached target Remote File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 6.404334] systemd[1]: Reached target Slice Units. vm-test-run-nixbot-gitlab> gitlab # [ 6.404742] systemd[1]: Reached target Swaps. vm-test-run-nixbot-gitlab> gitlab # [ 6.406842] systemd[1]: Listening on Query the User Interactively for a Password. vm-test-run-nixbot-gitlab> gitlab # [ 6.409056] systemd[1]: Listening on Process Core Dump Socket. vm-test-run-nixbot-gitlab> gitlab # [ 6.410730] systemd[1]: Listening on Credential Encryption/Decryption. vm-test-run-nixbot-gitlab> gitlab # [ 6.412731] systemd[1]: Listening on Factory Reset Management. vm-test-run-nixbot-gitlab> gitlab # [ 6.413436] systemd[1]: Listening on Hostname Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 6.417393] systemd[1]: Starting Journal Log Access Socket... vm-test-run-nixbot-gitlab> gitlab # [ 6.418683] systemd[1]: Listening on Journal Audit Socket. vm-test-run-nixbot-gitlab> gitlab # [ 6.420600] systemd[1]: Listening on Console Output Muting Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 6.421341] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. vm-test-run-nixbot-gitlab> gitlab # [ 6.421825] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os vm-test-run-nixbot-gitlab> gitlab # [ 6.422340] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 6.424933] systemd[1]: Listening on Disk Repartitioning Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 6.426356] systemd[1]: Listening on udev Control Socket. vm-test-run-nixbot-gitlab> gitlab # [ 6.426910] systemd[1]: Listening on udev Varlink Socket. vm-test-run-nixbot-gitlab> gitlab # [ 6.438670] systemd[1]: Mounting Huge Pages File System... vm-test-run-nixbot-gitlab> gitlab # [ 6.443500] systemd[1]: Mounting POSIX Message Queue File System... vm-test-run-nixbot-gitlab> gitlab # [ 6.446923] systemd[1]: Mounting Kernel Debug File System... vm-test-run-nixbot-gitlab> gitlab # [ 6.452345] systemd[1]: Mounting Kernel Trace File System... vm-test-run-nixbot-gitlab> gitlab # [ 6.461158] systemd[1]: Starting Create List of Static Device Nodes... vm-test-run-nixbot-gitlab> gitlab # [ 6.465444] systemd[1]: Starting Load Kernel Module 9pnet_virtio... vm-test-run-nixbot-gitlab> gitlab # [ 6.466091] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs vm-test-run-nixbot-gitlab> gitlab # [ 6.474190] systemd[1]: Mounting Kernel Configuration File System... vm-test-run-nixbot-gitlab> gitlab # [ 6.477126] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm vm-test-run-nixbot-gitlab> gitlab # [ 6.477771] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore vm-test-run-nixbot-gitlab> gitlab # [ 6.480931] systemd[1]: Starting Load Kernel Module fuse... vm-test-run-nixbot-gitlab> gitlab # [ 6.482231] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 vm-test-run-nixbot-gitlab> gitlab # [ 6.491373] systemd[1]: Starting Journal Service... vm-test-run-nixbot-gitlab> gitlab # [ 6.493789] systemd[1]: Starting Load Kernel Modules... vm-test-run-nixbot-gitlab> gitlab # [ 6.499674] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... vm-test-run-nixbot-gitlab> gitlab # [ 6.506946] systemd[1]: Starting Remount Root and Kernel File Systems... vm-test-run-nixbot-gitlab> gitlab # [ 6.508699] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os vm-test-run-nixbot-gitlab> gitlab # [ 6.515289] fuse: init (API version 7.45) vm-test-run-nixbot-gitlab> gitlab # [ 6.515834] systemd[1]: Starting Coldplug All udev Devices... vm-test-run-nixbot-gitlab> gitlab # [ 6.523222] systemd[1]: Listening on Journal Log Access Socket. vm-test-run-nixbot-gitlab> gitlab # [ 6.526662] systemd[1]: Mounted Huge Pages File System. vm-test-run-nixbot-gitlab> gitlab # [ 6.527480] systemd[1]: Mounted POSIX Message Queue File System. vm-test-run-nixbot-gitlab> gitlab # [ 6.528186] systemd[1]: Mounted Kernel Debug File System. vm-test-run-nixbot-gitlab> gitlab # [ 6.528763] systemd[1]: Mounted Kernel Trace File System. vm-test-run-nixbot-gitlab> gitlab # [ 6.530285] systemd[1]: Finished Create List of Static Device Nodes. vm-test-run-nixbot-gitlab> gitlab # [ 6.531461] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 6.532097] systemd-journald[315]: Collecting audit messages is enabled. vm-test-run-nixbot-gitlab> gitlab # [ 6.532442] systemd[1]: Finished Load Kernel Module 9pnet_virtio. vm-test-run-nixbot-gitlab> gitlab # [ 6.533190] systemd[1]: Mounted Kernel Configuration File System. vm-test-run-nixbot-gitlab> gitlab # [ 6.534014] systemd[1]: modprobe@fuse.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 6.534849] systemd[1]: Finished Load Kernel Module fuse. vm-test-run-nixbot-gitlab> gitlab # [ 6.535856] systemd[1]: Finished Load Kernel Modules. vm-test-run-nixbot-gitlab> gitlab # [ 6.541459] systemd[1]: Mounting FUSE Control File System... vm-test-run-nixbot-gitlab> gitlab # [ 6.549875] systemd[1]: Starting Firewall... vm-test-run-nixbot-gitlab> gitlab # [ 6.557281] EXT4-fs (vda): re-mounted b9118612-c3ed-4b3b-9aec-5cdb286e46a5. vm-test-run-nixbot-gitlab> gitlab # [ 6.557671] systemd[1]: Starting Apply Kernel Variables... vm-test-run-nixbot-gitlab> gitlab # [ 6.565397] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... vm-test-run-nixbot-gitlab> gitlab # [ 6.568481] systemd[1]: Finished Remount Root and Kernel File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 6.571054] systemd[1]: Mounted FUSE Control File System. vm-test-run-nixbot-gitlab> gitlab # [ 6.575132] systemd[1]: Queued start job for default target Multi-User System. vm-test-run-nixbot-gitlab> gitlab # [ 6.576836] systemd[1]: Unnecessary job was removed for /dev/vdb.[ 6.578059] systemd[1]: Started Journal Service. vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # [ 6.578675] systemd[1]: systemd-journald.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 6.580373] systemd-modules-load[316]: Using 3 probe threads vm-test-run-nixbot-gitlab> gitlab # [ 6.584407] systemd-modules-load[316]: Module 'atkbd' is built in vm-test-run-nixbot-gitlab> gitlab # [ 6.585848] systemd-modules-load[316]: Module 'loop' is built in vm-test-run-nixbot-gitlab> gitlab # [ 6.587147] systemd-modules-load[316]: Inserted module 'tls' vm-test-run-nixbot-gitlab> gitlab # [ 6.590257] systemd[1]: Listening on Disk Image Download Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 6.595441] systemd[1]: Starting Flush Journal to Persistent Storage... vm-test-run-nixbot-gitlab> gitlab # [ 6.597240] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore vm-test-run-nixbot-gitlab> gitlab # [ 6.604386] systemd[1]: Starting Load/Save OS Random Seed... vm-test-run-nixbot-gitlab> gitlab # [ 6.605569] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os vm-test-run-nixbot-gitlab> gitlab # [ 6.610449] systemd-oomd[317]: No swap; memory pressure usage will be degraded vm-test-run-nixbot-gitlab> gitlab # [ 6.617079] systemd-journald[315]: Received client request to flush runtime journal. vm-test-run-nixbot-gitlab> gitlab # [ 6.643514] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. vm-test-run-nixbot-gitlab> gitlab # [ 6.644927] systemd[1]: Finished Apply Kernel Variables. vm-test-run-nixbot-gitlab> gitlab # [ 6.646528] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. vm-test-run-nixbot-gitlab> gitlab # [ 6.648724] systemd[1]: Starting Create Static Device Nodes in /dev... vm-test-run-nixbot-gitlab> gitlab # [ 6.649942] systemd[1]: Finished Create Static Device Nodes in /dev. vm-test-run-nixbot-gitlab> gitlab # [ 6.652132] systemd[1]: Reached target Preparation for Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 6.653308] systemd[1]: Starting Rule-based Manager for Device Events and Files... vm-test-run-nixbot-gitlab> gitlab # [ 6.655078] systemd[1]: Finished Load/Save OS Random Seed. vm-test-run-nixbot-gitlab> gitlab # [ 6.656629] systemd[1]: Reached target First Boot Complete. vm-test-run-nixbot-gitlab> gitlab # [ 6.657503] systemd[1]: Finished Flush Journal to Persistent Storage. vm-test-run-nixbot-gitlab> gitlab # [ 6.691035] systemd-udevd[353]: Using default interface naming scheme 'v261'. vm-test-run-nixbot-gitlab> gitlab # [ 6.722274] systemd[1]: Started Rule-based Manager for Device Events and Files. vm-test-run-nixbot-gitlab> gitlab # [ 6.918606] systemd[1]: Finished Firewall. vm-test-run-nixbot-gitlab> gitlab # [ 7.155510] systemd[1]: Finished Coldplug All udev Devices. vm-test-run-nixbot-gitlab> gitlab # [ 7.201756] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs vm-test-run-nixbot-gitlab> gitlab # [ 7.247148] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. vm-test-run-nixbot-gitlab> gitlab # [ 7.257368] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. vm-test-run-nixbot-gitlab> gitlab # [ 7.282146] (udev-worker)[470]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. vm-test-run-nixbot-gitlab> gitlab # [ 7.284773] (udev-worker)[470]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 7.300456] systemd[1]: Condition check resulted in Virtio network device being skipped. vm-test-run-nixbot-gitlab> gitlab # [ 7.307280] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore vm-test-run-nixbot-gitlab> gitlab # [ 7.309632] (udev-worker)[402]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 7.310874] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 vm-test-run-nixbot-gitlab> gitlab # [ 7.313321] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore vm-test-run-nixbot-gitlab> gitlab # [ 7.314951] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os vm-test-run-nixbot-gitlab> gitlab # [ 7.317289] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os vm-test-run-nixbot-gitlab> gitlab # [ 7.360604] mousedev: PS/2 mouse device common for all mice vm-test-run-nixbot-gitlab> gitlab # [ 7.385705] systemd[1]: Mounting /run/wrappers... vm-test-run-nixbot-gitlab> gitlab # [ 7.412875] systemd[1]: Mounted /run/wrappers. vm-test-run-nixbot-gitlab> gitlab # [ 7.413963] systemd[1]: Reached target Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 7.415159] systemd[1]: Listening on Boot Loader Control Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.417535] systemd[1]: Starting register-nix-paths.service... vm-test-run-nixbot-gitlab> gitlab # [ 7.420810] systemd[1]: Starting Create SUID/SGID Wrappers... vm-test-run-nixbot-gitlab> gitlab # [ 7.422203] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. vm-test-run-nixbot-gitlab> gitlab # [ 7.428380] systemd[1]: Starting Save Transient machine-id to Disk... vm-test-run-nixbot-gitlab> gitlab # [ 7.436697] systemd[1]: Starting Create System Files and Directories... vm-test-run-nixbot-gitlab> gitlab # [ 7.459750] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.463249] systemd[1]: Finished Save Transient machine-id to Disk. vm-test-run-nixbot-gitlab> gitlab # [ 7.473826] systemd[1]: Finished Create System Files and Directories. vm-test-run-nixbot-gitlab> gitlab # [ 7.480234] systemd[1]: Starting Rebuild Journal Catalog... vm-test-run-nixbot-gitlab> gitlab # [ 7.489783] systemd[1]: Starting Record System Boot/Shutdown in UTMP... vm-test-run-nixbot-gitlab> gitlab # [ 7.509544] systemd[1]: Finished Record System Boot/Shutdown in UTMP. vm-test-run-nixbot-gitlab> gitlab # [ 7.531481] systemd[1]: Finished Rebuild Journal Catalog. vm-test-run-nixbot-gitlab> gitlab # [ 7.533542] systemd[1]: Starting Update is Completed... vm-test-run-nixbot-gitlab> gitlab # [ 7.548325] systemd[1]: Finished Update is Completed. vm-test-run-nixbot-gitlab> gitlab # [ 7.584281] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.585964] systemd[1]: Finished Create SUID/SGID Wrappers. vm-test-run-nixbot-gitlab> gitlab # [ 7.587411] systemd[1]: suid-sgid-wrappers.service: Consumed 134ms CPU time over 163ms wall clock time, 6.4M memory peak, 1.9M read from disk. vm-test-run-nixbot-gitlab> gitlab # [ 7.833514] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3 vm-test-run-nixbot-gitlab> gitlab # [ 7.849969] systemd[1]: Finished register-nix-paths.service. vm-test-run-nixbot-gitlab> gitlab # [ 7.856061] systemd[1]: Reached target System Initialization. vm-test-run-nixbot-gitlab> gitlab # [ 7.857038] systemd[1]: Started Discard unused filesystem blocks once a week. vm-test-run-nixbot-gitlab> gitlab # [ 7.858080] systemd[1]: Started logrotate.timer. vm-test-run-nixbot-gitlab> gitlab # [ 7.858795] systemd[1]: Started Daily Cleanup of Temporary Directories. vm-test-run-nixbot-gitlab> gitlab # [ 7.859760] systemd[1]: Reached target Timer Units. vm-test-run-nixbot-gitlab> gitlab # [ 7.860782] systemd[1]: Listening on D-Bus System Message Bus Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.861979] systemd[1]: Listening on Nix Daemon Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.872384] systemd[1]: Starting nixbot CI service socket... vm-test-run-nixbot-gitlab> gitlab # [ 7.873356] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.876664] systemd[1]: Starting D-Bus System Message Bus... vm-test-run-nixbot-gitlab> gitlab # [ 7.877630] systemd[1]: Listening on nixbot CI service socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.879276] systemd[1]: Reached target Socket Units. vm-test-run-nixbot-gitlab> gitlab # [ 7.921528] dbus-broker-launch[562]: Looking up NSS user entry for 'systemd-timesync'... vm-test-run-nixbot-gitlab> gitlab # [ 7.924397] dbus-broker-launch[562]: NSS returned no entry for 'systemd-timesync' vm-test-run-nixbot-gitlab> gitlab # [ 7.927855] dbus-broker-launch[562]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" vm-test-run-nixbot-gitlab> gitlab # [ 7.933835] systemd[1]: Started D-Bus System Message Bus. vm-test-run-nixbot-gitlab> gitlab # [ 7.946591] dbus-broker-launch[562]: Ready vm-test-run-nixbot-gitlab> gitlab # [ 7.956067] systemd[1]: Reached target Basic System. vm-test-run-nixbot-gitlab> gitlab # [ 7.959008] systemd[1]: Started backdoor.service. vm-test-run-nixbot-gitlab> gitlab # [ 7.961000] systemd[1]: Starting gitlab-config.service... vm-test-run-nixbot-gitlab> gitlab # [ 7.974699] systemd[1]: Starting Import lastlog data into lastlog2 database... vm-test-run-nixbot-gitlab> gitlab # [ 7.981966] systemd[1]: Starting Logrotate configuration check... vm-test-run-nixbot-gitlab> gitlab # [ 7.985711] systemd[1]: Starting Name Service Cache Daemon (nsncd)... vm-test-run-nixbot-gitlab> gitlab # connecting to host... vm-test-run-nixbot-gitlab> gitlab # [ 7.994095] systemd[1]: Starting Post-Boot Actions... vm-test-run-nixbot-gitlab> gitlab # [ 7.998760] systemd[1]: Started Reset console on configuration changes. vm-test-run-nixbot-gitlab> gitlab # [ 8.000654] logrotate[570]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. vm-test-run-nixbot-gitlab> gitlab # [ 8.002906] logrotate[570]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf vm-test-run-nixbot-gitlab> gitlab # [ 8.004476] logrotate[570]: note: 'monthly' overrides previously specified 'weekly' vm-test-run-nixbot-gitlab> gitlab # [ 8.005654] logrotate[570]: Reading state from file: /var/lib/logrotate.status vm-test-run-nixbot-gitlab> gitlab # [ 8.006665] logrotate[570]: state file /var/lib/logrotate.status does not exist vm-test-run-nixbot-gitlab> gitlab # [ 8.007701] logrotate[570]: Allocating hash table for state file, size 64 entries vm-test-run-nixbot-gitlab> gitlab: Guest shell says: b'Spawning backdoor root shell...\n' vm-test-run-nixbot-gitlab> gitlab # [ 8.012145] logrotate[570]: Handling 4 logs vm-test-run-nixbot-gitlab> gitlab # [ 8.012887] logrotate[570]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab: connected to guest root shell vm-test-run-nixbot-gitlab> gitlab # [ 8.014889] logrotate[570]: considering log /var/log/btmp vm-test-run-nixbot-gitlab> gitlab: (connecting took 8.46 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 8.017698] logrotate[570]: Creating new state vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for the VM to finish booting, in 8.46 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 8.018458] logrotate[570]: Now: 2026-10-04 11:26 vm-test-run-nixbot-gitlab> gitlab # [ 8.019114] logrotate[570]: Last rotated at 2026-10-04 11:00 vm-test-run-nixbot-gitlab> gitlab # [ 8.019916] logrotate[570]: log does not need rotating (log has already been rotated) vm-test-run-nixbot-gitlab> gitlab # [ 8.022353] logrotate[570]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 8.025086] logrotate[570]: considering log /var/log/wtmp vm-test-run-nixbot-gitlab> gitlab # [ 8.026074] logrotate[570]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 8.026674] logrotate[570]: Now: 2026-10-04 11:26 vm-test-run-nixbot-gitlab> gitlab # [ 8.028425] logrotate[570]: Last rotated at 2026-10-04 11:00 vm-test-run-nixbot-gitlab> gitlab # [ 8.030859] logrotate[570]: log does not need rotating (log has already been rotated) vm-test-run-nixbot-gitlab> gitlab # [ 8.032577] logrotate[570]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 8.034503] logrotate[570]: switching euid from 0 to 165 and egid from 0 to 165 (pid 570) vm-test-run-nixbot-gitlab> gitlab # [ 8.035612] logrotate[570]: considering log /var/gitlab/state/log/*.log vm-test-run-nixbot-gitlab> gitlab # [ 8.037665] logrotate[570]: log /var/gitlab/state/log/*.log does not exist -- skipping vm-test-run-nixbot-gitlab> gitlab # [ 8.038810] logrotate[570]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 8.039606] logrotate[570]: switching euid from 165 to 0 and egid from 165 to 0 (pid 570) vm-test-run-nixbot-gitlab> gitlab # [ 8.041119] logrotate[570]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 8.043906] logrotate[570]: switching euid from 0 to 60 and egid from 0 to 60 (pid 570) vm-test-run-nixbot-gitlab> gitlab # [ 8.045260] logrotate[570]: considering log /var/log/nginx/*.log vm-test-run-nixbot-gitlab> gitlab # [ 8.046238] logrotate[570]: log /var/log/nginx/*.log does not exist -- skipping vm-test-run-nixbot-gitlab> gitlab # [ 8.047300] logrotate[570]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 8.047914] logrotate[570]: not running postrotate script, since no logs were rotated vm-test-run-nixbot-gitlab> gitlab # [ 8.050417] logrotate[570]: switching euid from 60 to 0 and egid from 60 to 0 (pid 570) vm-test-run-nixbot-gitlab> gitlab # [ 8.051672] systemd[1]: Starting resolvconf update... vm-test-run-nixbot-gitlab> gitlab # [ 8.053606] nsncd[572]: Oct 04 11:26:29.921 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" vm-test-run-nixbot-gitlab> gitlab # [ 8.056953] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 8.058076] systemd[1]: Started Name Service Cache Daemon (nsncd). vm-test-run-nixbot-gitlab> gitlab # [ 8.059315] systemd[1]: Finished Logrotate configuration check. vm-test-run-nixbot-gitlab> gitlab # [ 8.060386] systemd[1]: Finished Post-Boot Actions. vm-test-run-nixbot-gitlab> gitlab # [ 8.061540] systemd[1]: Reached target Host and Network Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 8.062615] systemd[1]: Reached target User and Group Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 8.064400] systemd[1]: Starting User Login Management... vm-test-run-nixbot-gitlab> gitlab # [ 8.065345] systemd[1]: Finished Import lastlog data into lastlog2 database. vm-test-run-nixbot-gitlab> gitlab # [ 8.080750] systemd-vconsole-setup[581]: Configuration of first virtual console was skipped, ignoring remaining ones. vm-test-run-nixbot-gitlab> gitlab # [ 8.086573] systemd[1]: Finished Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 8.096934] systemd[1]: Stopped target Host and Network Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 8.098737] systemd[1]: Stopping Host and Network Name Lookups... vm-test-run-nixbot-gitlab> gitlab # [ 8.099761] systemd[1]: Stopped target User and Group Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 8.100865] systemd[1]: Stopping User and Group Name Lookups... vm-test-run-nixbot-gitlab> gitlab # [ 8.101782] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... vm-test-run-nixbot-gitlab> gitlab # [ 8.103592] systemd[1]: nscd.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 8.104615] systemd[1]: Stopped Name Service Cache Daemon (nsncd). vm-test-run-nixbot-gitlab> gitlab # [ 8.105512] systemd[1]: nscd.service: Consumed 27ms CPU time over 116ms wall clock time, 7.6M memory peak, 5.8M read from disk. vm-test-run-nixbot-gitlab> gitlab # [ 8.111138] systemd[1]: Starting Name Service Cache Daemon (nsncd)... vm-test-run-nixbot-gitlab> gitlab # [ 8.113024] systemd-logind[606]: New seat seat0. vm-test-run-nixbot-gitlab> gitlab # [ 8.117338] systemd-logind[606]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) vm-test-run-nixbot-gitlab> gitlab # [ 8.120681] systemd-logind[606]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) vm-test-run-nixbot-gitlab> gitlab # [ 8.121982] systemd-logind[606]: Watching system buttons on /dev/input/event0 (gpio-keys) vm-test-run-nixbot-gitlab> gitlab # [ 8.123142] systemd[1]: Started User Login Management. vm-test-run-nixbot-gitlab> gitlab # [ 8.132615] systemd[1]: Starting linger-users.service... vm-test-run-nixbot-gitlab> gitlab # [ 8.136961] systemd[1]: Finished resolvconf update. vm-test-run-nixbot-gitlab> gitlab # [ 8.139951] systemd[1]: Reached target Preparation for Network. vm-test-run-nixbot-gitlab> gitlab # [ 8.143831] nsncd[663]: Oct 04 11:26:30.041 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" vm-test-run-nixbot-gitlab> gitlab # [ 8.146142] systemd[1]: Starting DHCP Client... vm-test-run-nixbot-gitlab> gitlab # [ 8.147507] systemd[1]: Starting Address configuration of eth1... vm-test-run-nixbot-gitlab> gitlab # [ 8.151968] systemd[1]: Starting Extra networking commands.... vm-test-run-nixbot-gitlab> gitlab # [ 8.154461] systemd[1]: Started Name Service Cache Daemon (nsncd). vm-test-run-nixbot-gitlab> gitlab # [ 8.156727] systemd[1]: linger-users.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 8.158418] systemd[1]: Finished linger-users.service. vm-test-run-nixbot-gitlab> gitlab # [ 8.163029] systemd[1]: Reached target Host and Network Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 8.164807] systemd[1]: Reached target User and Group Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 8.178415] network-addresses-eth1-start[690]: adding address 192.168.1.1/24... done vm-test-run-nixbot-gitlab> gitlab # [ 8.183719] network-addresses-eth1-start[690]: adding address 2001:db8:1::1/64... done vm-test-run-nixbot-gitlab> gitlab # [ 8.192060] systemd[1]: Finished Address configuration of eth1. vm-test-run-nixbot-gitlab> gitlab # [ 8.233519] dhcpcd[708]: dhcpcd-10.3.2 starting vm-test-run-nixbot-gitlab> gitlab # [ 8.235713] dhcpcd[754]: dev: loaded udev vm-test-run-nixbot-gitlab> gitlab # [ 8.238893] systemd[1]: Finished Extra networking commands.. vm-test-run-nixbot-gitlab> gitlab # [ 8.240599] systemd[1]: Reached target Network. vm-test-run-nixbot-gitlab> gitlab # [ 8.244757] systemd[1]: Starting gitlab-workhorse.service... vm-test-run-nixbot-gitlab> gitlab # [ 8.248169] 8021q: 802.1Q VLAN Support v1.8 vm-test-run-nixbot-gitlab> gitlab # [ 8.248600] 8021q: adding VLAN 0 to HW filter on device eth1 vm-test-run-nixbot-gitlab> gitlab # [ 8.249207] systemd[1]: Starting Nginx Web Server... vm-test-run-nixbot-gitlab> gitlab # [ 8.257411] systemd[1]: Starting PostgreSQL Server... vm-test-run-nixbot-gitlab> gitlab # [ 8.262938] systemd[1]: Starting Redis Server - redis-gitlab... vm-test-run-nixbot-gitlab> gitlab # [ 8.267461] systemd[1]: Starting Permit User Sessions... vm-test-run-nixbot-gitlab> gitlab # [ 8.277954] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. vm-test-run-nixbot-gitlab> gitlab # [ 8.294692] cfg80211: Loading compiled-in X.509 certificates for regulatory database vm-test-run-nixbot-gitlab> gitlab # [ 8.296780] systemd[1]: Finished Permit User Sessions. vm-test-run-nixbot-gitlab> gitlab # [ 8.303125] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' vm-test-run-nixbot-gitlab> gitlab # [ 8.303588] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' vm-test-run-nixbot-gitlab> gitlab # [ 8.306063] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 vm-test-run-nixbot-gitlab> gitlab # [ 8.306391] cfg80211: failed to load regulatory.db vm-test-run-nixbot-gitlab> gitlab # [ 8.303491] systemd[1]: Started Getty on tty1. vm-test-run-nixbot-gitlab> gitlab # [ 8.308878] systemd[1]: Reached target Login Prompts. vm-test-run-nixbot-gitlab> gitlab # [ 8.336486] 8021q: adding VLAN 0 to HW filter on device eth0 vm-test-run-nixbot-gitlab> gitlab # [ 8.336190] dhcpcd[754]: eth0: waiting for carrier vm-test-run-nixbot-gitlab> gitlab # [ 8.338354] dhcpcd[754]: eth0: carrier acquired vm-test-run-nixbot-gitlab> gitlab # [ 8.342680] dhcpcd[754]: DUID 00:01:00:01:32:54:f4:66:52:54:00:12:34:56 vm-test-run-nixbot-gitlab> gitlab # [ 8.344242] dhcpcd[754]: eth0: IAID 00:12:34:56 vm-test-run-nixbot-gitlab> gitlab # [ 8.345081] dhcpcd[754]: eth0: adding address fe80::5054:ff:fe12:3456 vm-test-run-nixbot-gitlab> gitlab # [ 8.418449] nginx-pre-start[792]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok vm-test-run-nixbot-gitlab> gitlab # [ 8.420884] nginx-pre-start[792]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful vm-test-run-nixbot-gitlab> gitlab # [ 8.423735] systemd[1]: Started Nginx Web Server. vm-test-run-nixbot-gitlab> gitlab # [ 8.448761] redis-server[790]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. vm-test-run-nixbot-gitlab> gitlab # [ 8.451238] redis-server[790]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo vm-test-run-nixbot-gitlab> gitlab # [ 8.453118] redis-server[790]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=790, just started vm-test-run-nixbot-gitlab> gitlab # [ 8.456639] redis-server[790]: Configuration loaded vm-test-run-nixbot-gitlab> gitlab # [ 8.457542] systemd[1]: Started Redis Server - redis-gitlab. vm-test-run-nixbot-gitlab> gitlab # [ 8.458808] postgresql-pre-start[794]: The files belonging to this database system will be owned by user "postgres". vm-test-run-nixbot-gitlab> gitlab # [ 8.460492] postgresql-pre-start[794]: This user must also own the server process. vm-test-run-nixbot-gitlab> gitlab # [ 8.461646] postgresql-pre-start[794]: The database cluster will be initialized with locale "en_US.UTF-8". vm-test-run-nixbot-gitlab> gitlab # [ 8.463939] postgresql-pre-start[794]: The default database encoding has accordingly been set to "UTF8". vm-test-run-nixbot-gitlab> gitlab # [ 8.465451] postgresql-pre-start[794]: The default text search configuration will be set to "english". vm-test-run-nixbot-gitlab> gitlab # [ 8.466793] postgresql-pre-start[794]: Data page checksums are enabled. vm-test-run-nixbot-gitlab> gitlab # [ 8.467749] postgresql-pre-start[794]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok vm-test-run-nixbot-gitlab> gitlab # [ 8.469206] postgresql-pre-start[794]: creating subdirectories ... ok vm-test-run-nixbot-gitlab> gitlab # [ 8.470220] postgresql-pre-start[794]: selecting dynamic shared memory implementation ... posix vm-test-run-nixbot-gitlab> gitlab # [ 8.471533] redis[790]: monotonic clock: ARM CNTVCT @ 1000 ticks/us vm-test-run-nixbot-gitlab> gitlab # [ 8.472874] redis[790]: Running mode=standalone, port=0. vm-test-run-nixbot-gitlab> gitlab # [ 8.473839] redis[790]: Server initialized vm-test-run-nixbot-gitlab> gitlab # [ 8.474604] redis[790]: Ready to accept connections unix vm-test-run-nixbot-gitlab> gitlab # [ 8.475521] redis[790]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. vm-test-run-nixbot-gitlab> gitlab # [ 8.502999] postgresql-pre-start[794]: selecting default "max_connections" ... 100 vm-test-run-nixbot-gitlab> gitlab # [ 8.551348] postgresql-pre-start[794]: selecting default "shared_buffers" ... 128MB vm-test-run-nixbot-gitlab> gitlab # [ 8.868509] systemd[1]: Started gitlab-workhorse.service. vm-test-run-nixbot-gitlab> gitlab # [ 8.999883] postgresql-pre-start[794]: selecting default time zone ... UTC vm-test-run-nixbot-gitlab> gitlab # [ 9.002854] postgresql-pre-start[794]: creating configuration files ... ok vm-test-run-nixbot-gitlab> gitlab # [ 9.057279] gitlab-workhorse[817]: time="2026-10-04T11:26:30Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 vm-test-run-nixbot-gitlab> gitlab # [ 9.059076] gitlab-workhorse[817]: time="2026-10-04T11:26:30Z" level=info msg="Using redis/go-redis" vm-test-run-nixbot-gitlab> gitlab # [ 9.060503] gitlab-workhorse[817]: time="2026-10-04T11:26:30Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix vm-test-run-nixbot-gitlab> gitlab # [ 9.266685] postgresql-pre-start[794]: running bootstrap script ... ok vm-test-run-nixbot-gitlab> gitlab # [ 9.845567] dhcpcd[754]: eth0: soliciting a DHCP lease vm-test-run-nixbot-gitlab> gitlab # [ 9.864574] dhcpcd[754]: eth0: offered 10.0.2.15 from 10.0.2.2 vm-test-run-nixbot-gitlab> gitlab # [ 9.896297] dhcpcd[754]: eth0: probing address 10.0.2.15/24 vm-test-run-nixbot-gitlab> gitlab # [ 9.909736] postgresql-pre-start[794]: performing post-bootstrap initialization ... ok vm-test-run-nixbot-gitlab> gitlab # [ 10.063118] dhcpcd[754]: eth0: soliciting an IPv6 router vm-test-run-nixbot-gitlab> gitlab # [ 10.065279] dhcpcd[754]: eth0: Router Advertisement from fe80::2 vm-test-run-nixbot-gitlab> gitlab # [ 10.067132] dhcpcd[754]: eth0: adding address fec0::5054:ff:fe12:3456/64 vm-test-run-nixbot-gitlab> gitlab # [ 10.069018] dhcpcd[754]: eth0: adding route to fec0::/64 vm-test-run-nixbot-gitlab> gitlab # [ 10.070499] dhcpcd[754]: eth0: adding default route via fe80::2 vm-test-run-nixbot-gitlab> gitlab # [ 10.077925] postgresql-pre-start[794]: syncing data to disk ... ok vm-test-run-nixbot-gitlab> gitlab # [ 10.079303] postgresql-pre-start[794]: initdb: warning: enabling "trust" authentication for local connections vm-test-run-nixbot-gitlab> gitlab # [ 10.081567] postgresql-pre-start[794]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. vm-test-run-nixbot-gitlab> gitlab # [ 10.084943] postgresql-pre-start[794]: Success. You can now start the database server using: vm-test-run-nixbot-gitlab> gitlab # [ 10.086802] postgresql-pre-start[794]: pg_ctl -D /var/lib/postgresql/18 -l logfile start vm-test-run-nixbot-gitlab> gitlab # [ 10.237414] postgres[842]: [842] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit vm-test-run-nixbot-gitlab> gitlab # [ 10.240392] postgres[842]: [842] LOG: listening on IPv6 address "::1", port 5432 vm-test-run-nixbot-gitlab> gitlab # [ 10.242192] postgres[842]: [842] LOG: listening on IPv4 address "127.0.0.1", port 5432 vm-test-run-nixbot-gitlab> gitlab # [ 10.244103] postgres[842]: [842] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" vm-test-run-nixbot-gitlab> gitlab # [ 10.250526] postgres[860]: [860] LOG: database system was shut down at 2026-10-04 11:26:31 GMT vm-test-run-nixbot-gitlab> gitlab # [ 10.260959] postgres[842]: [842] LOG: database system is ready to accept connections vm-test-run-nixbot-gitlab> gitlab # [ 10.264309] systemd[1]: Started PostgreSQL Server. vm-test-run-nixbot-gitlab> gitlab # [ 10.270263] systemd[1]: Starting PostgreSQL Setup Scripts... vm-test-run-nixbot-gitlab> gitlab # [ 10.533237] postgresql-setup-start[871]: CREATE DATABASE vm-test-run-nixbot-gitlab> gitlab # [ 10.566276] postgresql-setup-start[881]: CREATE ROLE vm-test-run-nixbot-gitlab> gitlab # [ 10.579702] postgresql-setup-start[883]: ALTER DATABASE vm-test-run-nixbot-gitlab> gitlab # [ 10.612413] postgresql-setup-start[888]: CREATE ROLE vm-test-run-nixbot-gitlab> gitlab # [ 10.614826] systemd[1]: Finished PostgreSQL Setup Scripts. vm-test-run-nixbot-gitlab> gitlab # [ 10.617077] systemd[1]: Reached target PostgreSQL. vm-test-run-nixbot-gitlab> gitlab # [ 10.620124] systemd[1]: Starting gitlab-postgresql.service... vm-test-run-nixbot-gitlab> gitlab # [ 10.677245] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[876]: mkdir -p /var/gitlab/state/home/.ssh: OK vm-test-run-nixbot-gitlab> gitlab # [ 10.682020] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[876]: chmod 700 /var/gitlab/state/home/.ssh: OK vm-test-run-nixbot-gitlab> gitlab # [ 10.749529] systemd[1]: Finished gitlab-config.service. vm-test-run-nixbot-gitlab> gitlab # [ 10.753095] systemd[1]: Started gitaly.service. vm-test-run-nixbot-gitlab> gitlab # [ 10.759713] systemd[1]: Starting gitlab-db-config.service... vm-test-run-nixbot-gitlab> gitlab # [ 10.761609] gitlab-postgresql-start[898]: CREATE DATABASE vm-test-run-nixbot-gitlab> gitlab # [ 10.805591] gitlab-postgresql-start[914]: CREATE EXTENSION vm-test-run-nixbot-gitlab> gitlab # [ 10.905811] gitlab-postgresql-start[918]: CREATE EXTENSION vm-test-run-nixbot-gitlab> gitlab # [ 10.911436] systemd[1]: Finished gitlab-postgresql.service. vm-test-run-nixbot-gitlab> gitlab # [ 11.332276] gitaly[910]: time="2026-10-04T11:26:33.230Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=910 vm-test-run-nixbot-gitlab> gitlab # [ 11.344258] gitaly[910]: time="2026-10-04T11:26:33.231Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=910 vm-test-run-nixbot-gitlab> gitlab # [ 11.346041] gitaly[910]: time="2026-10-04T11:26:33.231Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=910 vm-test-run-nixbot-gitlab> gitlab # [ 11.348338] gitaly[910]: time="2026-10-04T11:26:33.231Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=910 vm-test-run-nixbot-gitlab> gitlab # [ 11.350236] gitaly[910]: time="2026-10-04T11:26:33.231Z" level=info msg="Starting Gitaly" pid=910 version= vm-test-run-nixbot-gitlab> gitlab # [ 11.533921] gitaly[910]: time="2026-10-04T11:26:33.432Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=201 pid=910 vm-test-run-nixbot-gitlab> gitlab # [ 11.537164] gitaly[910]: time="2026-10-04T11:26:33.435Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=910 vm-test-run-nixbot-gitlab> gitlab # [ 11.539390] gitaly[910]: time="2026-10-04T11:26:33.435Z" level=info msg="finished initializing command factory" duration_ms=0 pid=910 vm-test-run-nixbot-gitlab> gitlab # [ 11.541790] gitaly[910]: time="2026-10-04T11:26:33.435Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-910/git-exec-2184122490.d/git pid=910 vm-test-run-nixbot-gitlab> gitlab # [ 11.546340] gitaly[910]: time="2026-10-04T11:26:33.445Z" level=info msg="finished detecting git version" duration_ms=9 pid=910 vm-test-run-nixbot-gitlab> gitlab # [ 11.549358] gitaly[910]: time="2026-10-04T11:26:33.445Z" level=info msg="using Git version" pid=910 version=2.53.g vm-test-run-nixbot-gitlab> gitlab # [ 11.550964] gitaly[910]: time="2026-10-04T11:26:33.446Z" level=info msg="clearing disk cache object folder" pid=910 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 11.552850] gitaly[910]: time="2026-10-04T11:26:33.447Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache392049825 pid=910 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 11.556292] gitaly[910]: time="2026-10-04T11:26:33.447Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache392049825 pid=910 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 11.559837] gitaly[910]: time="2026-10-04T11:26:33.447Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=910 vm-test-run-nixbot-gitlab> gitlab # [ 11.563287] gitaly[910]: time="2026-10-04T11:26:33.447Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=910 vm-test-run-nixbot-gitlab> gitlab # [ 11.566634] gitaly[910]: time="2026-10-04T11:26:33.447Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache392049825 clear_duration_ms=0 pid=910 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 11.572907] gitaly[910]: time="2026-10-04T11:26:33.471Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=910 vm-test-run-nixbot-gitlab> gitlab # [ 11.575698] gitaly[910]: time="2026-10-04T11:26:33.472Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=910 schema=unix vm-test-run-nixbot-gitlab> gitlab # [ 11.577879] gitaly[910]: time="2026-10-04T11:26:33.472Z" level=info msg="listening at address" address=/run/gitaly/gitaly-910/sock.d/intern pid=910 schema=unix vm-test-run-nixbot-gitlab> gitlab # [ 11.580523] gitaly[910]: time="2026-10-04T11:26:33.479Z" level=info msg="counting repositories" pid=910 vm-test-run-nixbot-gitlab> gitlab # [ 11.581839] gitaly[910]: time="2026-10-04T11:26:33.479Z" level=info msg="starting to count repositories" pid=910 storage_path=/var/gitlab/state/repositories vm-test-run-nixbot-gitlab> gitlab # [ 11.583828] gitaly[910]: time="2026-10-04T11:26:33.479Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=910 vm-test-run-nixbot-gitlab> gitlab # [ 11.586091] gitaly[910]: time="2026-10-04T11:26:33.479Z" level=info msg="Started Gitaly" duration_ms=248 pid=910 vm-test-run-nixbot-gitlab> gitlab # [ 11.587905] gitaly[910]: time="2026-10-04T11:26:33.479Z" level=info msg="finished tempdir cleaner walk" pid=910 storage=default time_ms=0 vm-test-run-nixbot-gitlab> gitlab # [ 11.590293] gitaly[910]: time="2026-10-04T11:26:33.479Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=910 vm-test-run-nixbot-gitlab> gitlab # [ 11.592617] gitaly[910]: time="2026-10-04T11:26:33.479Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=910 storage_path=/var/gitlab/state/repositories vm-test-run-nixbot-gitlab> gitlab # [ 11.595364] gitaly[910]: time="2026-10-04T11:26:33.479Z" level=info msg="completed counting all repositories" counting_duration=0 pid=910 vm-test-run-nixbot-gitlab> gitlab # [ 11.597649] gitaly[910]: time="2026-10-04T11:26:33.480Z" level=info msg="maintenance: daily scheduled" pid=910 scheduled="2026-10-04 12:00:00 +0000 UTC" vm-test-run-nixbot-gitlab> gitlab # [ 11.918348] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[917]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 11.923406] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[917]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 11.931340] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[917]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 14.623896] dhcpcd[754]: eth0: leased 10.0.2.15 for 86400 seconds vm-test-run-nixbot-gitlab> gitlab # [ 14.624138] dhcpcd[754]: eth0: adding route to 10.0.2.0/24 vm-test-run-nixbot-gitlab> gitlab # [ 14.624206] dhcpcd[754]: eth0: adding default route via 10.0.2.2 vm-test-run-nixbot-gitlab> gitlab # [ 14.732109] systemd[1]: Started DHCP Client. vm-test-run-nixbot-gitlab> gitlab # [ 14.733523] systemd[1]: Reached target Network is Online. vm-test-run-nixbot-gitlab> gitlab # [ 14.975085] gitaly[910]: time="2026-10-04T11:26:36.873Z" level=info msg="License database preloaded" duration_ms=3642 pid=910 vm-test-run-nixbot-gitlab> gitlab # [ 57.337620] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[917]: W, [2026-10-04T11:27:19.235745 #917] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 59.131518] gitlab-workhorse[817]: time="2026-10-04T11:27:21Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 60.410075] postgres[1445]: [1445] ERROR: relation "feature_gates" does not exist at character 523 vm-test-run-nixbot-gitlab> gitlab # [ 60.412300] postgres[1445]: [1445] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), vm-test-run-nixbot-gitlab> gitlab # [ 60.414286] postgres[1445]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, vm-test-run-nixbot-gitlab> gitlab # [ 60.416426] postgres[1445]: c.collname, col_description(a.attrelid, a.attnum) AS comment, vm-test-run-nixbot-gitlab> gitlab # [ 60.418315] postgres[1445]: attidentity AS identity, vm-test-run-nixbot-gitlab> gitlab # [ 60.419610] postgres[1445]: attgenerated as attgenerated vm-test-run-nixbot-gitlab> gitlab # [ 60.420971] postgres[1445]: FROM pg_attribute a vm-test-run-nixbot-gitlab> gitlab # [ 60.422057] postgres[1445]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum vm-test-run-nixbot-gitlab> gitlab # [ 60.424031] postgres[1445]: LEFT JOIN pg_type t ON a.atttypid = t.oid vm-test-run-nixbot-gitlab> gitlab # [ 60.425477] postgres[1445]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation vm-test-run-nixbot-gitlab> gitlab # [ 60.427688] postgres[1445]: WHERE a.attrelid = '"feature_gates"'::regclass vm-test-run-nixbot-gitlab> gitlab # [ 60.429238] postgres[1445]: AND a.attnum > 0 AND NOT a.attisdropped vm-test-run-nixbot-gitlab> gitlab # [ 60.430723] postgres[1445]: ORDER BY a.attnum vm-test-run-nixbot-gitlab> gitlab # [ 60.431747] postgres[1445]: vm-test-run-nixbot-gitlab> gitlab # [ 66.281724] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1505]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping vm-test-run-nixbot-gitlab> gitlab # [ 66.284324] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1505]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping vm-test-run-nixbot-gitlab> gitlab # [ 69.139181] gitlab-workhorse[817]: time="2026-10-04T11:27:31Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 79.145921] gitlab-workhorse[817]: time="2026-10-04T11:27:41Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 86.950415] postgres[1601]: [1601] ERROR: relation "table_without_models" does not exist vm-test-run-nixbot-gitlab> gitlab # [ 86.950623] postgres[1601]: [1601] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') vm-test-run-nixbot-gitlab> gitlab # [ 89.153140] gitlab-workhorse[817]: time="2026-10-04T11:27:51Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 99.154336] gitlab-workhorse[817]: time="2026-10-04T11:28:01Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 104.661022] systemd[1]: Finished gitlab-db-config.service. vm-test-run-nixbot-gitlab> gitlab # [ 104.664625] systemd[1]: Started gitlab-sidekiq.service. vm-test-run-nixbot-gitlab> gitlab # [ 104.667944] systemd[1]: Starting gitlab.service... vm-test-run-nixbot-gitlab> gitlab # [ 105.217513] bundle[1849]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 105.220990] bundle[1849]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 105.224049] bundle[1849]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 109.161351] gitlab-workhorse[817]: time="2026-10-04T11:28:11Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 109.922125] sidekiq-cluster[1848]: {"severity":"INFO","time":"2026-10-04T11:28:11.779Z","message":"Starting cluster with 1 processes"} vm-test-run-nixbot-gitlab> gitlab # [ 109.923418] bundle[1849]: {"timestamp":"2026-10-04T11:28:11.792Z","pid":1849,"message":"* Enabling systemd notification integration"} vm-test-run-nixbot-gitlab> gitlab # [ 109.925231] bundle[1849]: {"timestamp":"2026-10-04T11:28:11.822Z","pid":1849,"message":"Puma starting in cluster mode..."} vm-test-run-nixbot-gitlab> gitlab # [ 109.928342] bundle[1849]: {"timestamp":"2026-10-04T11:28:11.822Z","pid":1849,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} vm-test-run-nixbot-gitlab> gitlab # [ 109.929935] bundle[1849]: {"timestamp":"2026-10-04T11:28:11.822Z","pid":1849,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} vm-test-run-nixbot-gitlab> gitlab # [ 109.931895] bundle[1849]: {"timestamp":"2026-10-04T11:28:11.822Z","pid":1849,"message":"* Min threads: 0"} vm-test-run-nixbot-gitlab> gitlab # [ 109.933300] bundle[1849]: {"timestamp":"2026-10-04T11:28:11.822Z","pid":1849,"message":"* Max threads: 4"} vm-test-run-nixbot-gitlab> gitlab # [ 109.934612] bundle[1849]: {"timestamp":"2026-10-04T11:28:11.822Z","pid":1849,"message":"* Environment: production"} vm-test-run-nixbot-gitlab> gitlab # [ 109.935976] bundle[1849]: {"timestamp":"2026-10-04T11:28:11.822Z","pid":1849,"message":"* Master PID: 1849"} vm-test-run-nixbot-gitlab> gitlab # [ 109.937386] bundle[1849]: {"timestamp":"2026-10-04T11:28:11.822Z","pid":1849,"message":"* Workers: 2"} vm-test-run-nixbot-gitlab> gitlab # [ 109.938660] bundle[1849]: {"timestamp":"2026-10-04T11:28:11.822Z","pid":1849,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} vm-test-run-nixbot-gitlab> gitlab # [ 109.940301] bundle[1849]: {"timestamp":"2026-10-04T11:28:11.822Z","pid":1849,"message":"* Preloading application"} vm-test-run-nixbot-gitlab> gitlab # [ 119.171130] gitlab-workhorse[817]: time="2026-10-04T11:28:21Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 129.180891] gitlab-workhorse[817]: time="2026-10-04T11:28:31Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 139.189353] gitlab-workhorse[817]: time="2026-10-04T11:28:41Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 148.266842] bundle[1849]: W, [2026-10-04T11:28:50.165120 #1849] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 149.190136] gitlab-workhorse[817]: time="2026-10-04T11:28:51Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 152.802245] sidekiq-cluster[1899]: 2026-10-04T11:28:54.700Z pid=1899 tid=wqr INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1899, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>434196480} vm-test-run-nixbot-gitlab> gitlab # [ 153.100586] sidekiq-cluster[1899]: W, [2026-10-04T11:28:54.999441 #1899] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 153.160510] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:55.059Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} vm-test-run-nixbot-gitlab> gitlab # [ 156.989477] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.887Z","message":"GitLab reliable fetch activated!"} vm-test-run-nixbot-gitlab> gitlab # [ 156.989712] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.888Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} vm-test-run-nixbot-gitlab> gitlab # [ 157.023662] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.922Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.027592] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.924Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.031516] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.927Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.035134] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.929Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.039150] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.932Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.042740] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.934Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.046356] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.936Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.049672] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.938Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.052971] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.940Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.056285] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.942Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.059871] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.944Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.063377] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.946Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.066519] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.948Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.069699] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.950Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.072649] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.951Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.075493] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.953Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.078268] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.956Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.080830] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.958Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.083538] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.961Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.086215] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.964Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.088956] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.966Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.091774] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.968Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.094335] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.970Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.096867] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.973Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.099305] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.975Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.101679] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.977Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.103790] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.980Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.106147] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.982Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.108340] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.985Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.110608] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.988Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.112788] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.991Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.115128] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.994Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.117340] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.996Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.119630] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:58.999Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.121988] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.001Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.124198] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.004Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.126413] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.007Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.128567] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.008Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.130680] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.011Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.132818] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.013Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.135042] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.015Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.137296] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.017Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.139497] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.019Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.141743] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.021Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.143967] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.024Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.146599] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.026Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.148883] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.028Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.151062] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.030Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.153324] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.033Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.155502] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.035Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.157816] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.037Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.159936] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.040Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.162334] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.052Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.164818] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.055Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.167083] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.057Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.170352] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.060Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.172637] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.062Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.174967] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.065Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.177326] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.067Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.179698] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.070Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.181950] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.072Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.184106] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.074Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.186344] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.077Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.188674] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.080Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.191021] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.083Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.193390] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.086Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.195559] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.088Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.197970] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.089Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.200365] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.091Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.202766] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.093Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.205144] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.095Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.207416] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.097Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.209815] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.099Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.212207] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.101Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.214539] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.102Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.216964] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.104Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.219186] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.106Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.221855] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.108Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.224115] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.110Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.226319] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.112Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.228800] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.114Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.231299] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.116Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.233808] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.119Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.236079] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.121Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.238452] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.123Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.240942] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.125Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.243311] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.128Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.245743] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.130Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.248112] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.132Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.250414] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.134Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.252801] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.135Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.255093] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.137Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.257553] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.141Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.260056] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.144Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.262113] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.147Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.264547] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.151Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.266875] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.154Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.269261] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.157Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.272526] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.171Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.274785] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.173Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.277233] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.176Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.281022] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.179Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.284175] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.183Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.286963] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.185Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.289777] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.188Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.292210] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.190Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.294389] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.192Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.358326] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.255Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 157.362031] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:28:59.259Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 159.190848] gitlab-workhorse[817]: time="2026-10-04T11:29:01Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 169.191653] gitlab-workhorse[817]: time="2026-10-04T11:29:11Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 179.201402] gitlab-workhorse[817]: time="2026-10-04T11:29:21Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 189.203763] gitlab-workhorse[817]: time="2026-10-04T11:29:31Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 194.067635] postgres[2646]: [2646] ERROR: relation "table_without_models" does not exist vm-test-run-nixbot-gitlab> gitlab # [ 194.070938] postgres[2646]: [2646] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') vm-test-run-nixbot-gitlab> gitlab # [ 196.998502] postgres[2676]: [2676] ERROR: relation "table_without_models" does not exist vm-test-run-nixbot-gitlab> gitlab # [ 196.998692] postgres[2676]: [2676] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') vm-test-run-nixbot-gitlab> gitlab # [ 199.209493] gitlab-workhorse[817]: time="2026-10-04T11:29:41Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 199.217970] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:41.116Z","message":"Booted Rails 7.2.3.1 application in production environment"} vm-test-run-nixbot-gitlab> gitlab # [ 199.225203] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:41.123Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} vm-test-run-nixbot-gitlab> gitlab # [ 199.229177] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:41.123Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} vm-test-run-nixbot-gitlab> gitlab # [ 199.231982] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:41.123Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} vm-test-run-nixbot-gitlab> gitlab # [ 199.236302] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:41.129Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} vm-test-run-nixbot-gitlab> gitlab # [ 201.978562] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:43.876Z","message":"Cleaning working queues"} vm-test-run-nixbot-gitlab> gitlab # [ 201.980462] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:43.878Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"3ce74fe4f940be7e1dc840c1","created_at":"2026-10-04T11:28:06.074Z","trace_propagation_headers":{"sentry-trace":"10c4894e8f774f91adc17b8b49c80d46-98c432aefe1641c8","baggage":"sentry-trace_id=10c4894e8f774f91adc17b8b49c80d46,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"a12081c9ce8a0f12abc03b58133d9fdd","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-04T11:28:06.085Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-3ce74fe4f940be7e1dc840c1: start","job_status":"start","queue_duration_s":97.792875,"scheduling_latency_s":97.792947} vm-test-run-nixbot-gitlab> gitlab # [ 202.237742] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:44.135Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"3ce74fe4f940be7e1dc840c1","created_at":"2026-10-04T11:28:06.074Z","trace_propagation_headers":{"sentry-trace":"10c4894e8f774f91adc17b8b49c80d46-98c432aefe1641c8","baggage":"sentry-trace_id=10c4894e8f774f91adc17b8b49c80d46,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"a12081c9ce8a0f12abc03b58133d9fdd","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-04T11:28:06.085Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-3ce74fe4f940be7e1dc840c1: done: 0.257334 sec","job_status":"done","queue_duration_s":97.792875,"scheduling_latency_s":97.792947,"redis_calls":29,"redis_duration_s":0.011795,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.006685,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00376,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00135,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.135978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.13461908,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.257334,"completed_at":"2026-10-04T11:29:44.135Z","load_balancing_strategy":"primary","db_duration_s":0.064066,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 203.462230] bundle[1849]: {"timestamp":"2026-10-04T11:29:45.360Z","pid":1849,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} vm-test-run-nixbot-gitlab> gitlab # [ 203.462494] bundle[1849]: {"timestamp":"2026-10-04T11:29:45.360Z","pid":1849,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} vm-test-run-nixbot-gitlab> gitlab # [ 203.462568] bundle[1849]: {"timestamp":"2026-10-04T11:29:45.360Z","pid":1849,"message":"! #\u003cThread:0x0000f0cca93d4b38 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} vm-test-run-nixbot-gitlab> gitlab # [ 203.462653] bundle[1849]: {"timestamp":"2026-10-04T11:29:45.360Z","pid":1849,"message":"! #\u003cThread:0x0000f0cca95a8298 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} vm-test-run-nixbot-gitlab> gitlab # [ 203.462713] bundle[1849]: {"timestamp":"2026-10-04T11:29:45.361Z","pid":1849,"message":"Use Ctrl-C to stop"} vm-test-run-nixbot-gitlab> gitlab # [ 203.462778] bundle[1849]: {"timestamp":"2026-10-04T11:29:45.361Z","pid":1849,"message":"Sending status to systemd every 1.0 sec"} vm-test-run-nixbot-gitlab> gitlab # [ 204.098496] bundle[1849]: {"timestamp":"2026-10-04T11:29:45.997Z","pid":1849,"message":"- Worker 0 (PID: 2775) booted in 0.6s, phase: 0"} vm-test-run-nixbot-gitlab> gitlab # [ 204.128548] bundle[1849]: {"timestamp":"2026-10-04T11:29:46.027Z","pid":1849,"message":"- Worker 1 (PID: 2779) booted in 0.61s, phase: 0"} vm-test-run-nixbot-gitlab> gitlab # [ 204.132491] systemd[1]: Started gitlab.service. vm-test-run-nixbot-gitlab> gitlab # [ 204.134253] systemd[1]: Reached target Common target for all GitLab services.. vm-test-run-nixbot-gitlab> gitlab # [ 204.135870] systemd[1]: Starting setup-gitlab.service... vm-test-run-nixbot-gitlab> gitlab # [ 204.168772] setup-gitlab-start[2803]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' vm-test-run-nixbot-gitlab> gitlab # [ 208.579460] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:29:50 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 4384 vm-test-run-nixbot-gitlab> gitlab # [ 208.583780] setup-gitlab-start[2803]: + TOKEN=nixbot-test-token-of-20-chars vm-test-run-nixbot-gitlab> gitlab # [ 208.585590] setup-gitlab-start[2803]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' vm-test-run-nixbot-gitlab> gitlab # [ 208.594841] runuser[2833]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) vm-test-run-nixbot-gitlab> gitlab # [ 209.141673] setup-gitlab-start[2834]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 209.145219] setup-gitlab-start[2834]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 209.148570] setup-gitlab-start[2834]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 209.982877] setup-gitlab-start[2834]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 209.983182] setup-gitlab-start[2834]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 209.985658] setup-gitlab-start[2834]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 216.613357] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:58.511Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"5b66a2cb4ac3b7506459d4e4","created_at":"2026-10-04T11:29:58.436Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e8f53ab1c191c6d15c5265e6d6fc32e4","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.508Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-5b66a2cb4ac3b7506459d4e4: start","job_status":"start","queue_duration_s":0.002473,"scheduling_latency_s":0.002535} vm-test-run-nixbot-gitlab> gitlab # [ 216.827363] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:58.725Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"5b66a2cb4ac3b7506459d4e4","created_at":"2026-10-04T11:29:58.436Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e8f53ab1c191c6d15c5265e6d6fc32e4","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.508Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-5b66a2cb4ac3b7506459d4e4: done: 0.214753 sec","job_status":"done","queue_duration_s":0.002473,"scheduling_latency_s":0.002535,"redis_calls":16,"redis_duration_s":0.017645,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.009652,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006616,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001377,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.064,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.064,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046758,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00149774,"duration_s":0.214753,"completed_at":"2026-10-04T11:29:58.725Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.08244,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 216.892510] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:58.741Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"b71b3ba29e59fd5fafa67c87","created_at":"2026-10-04T11:29:58.542Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.554Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-b71b3ba29e59fd5fafa67c87: start","job_status":"start","queue_duration_s":0.187008,"scheduling_latency_s":0.18704} vm-test-run-nixbot-gitlab> gitlab # [ 217.264924] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.163Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"b71b3ba29e59fd5fafa67c87","created_at":"2026-10-04T11:29:58.542Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.554Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-b71b3ba29e59fd5fafa67c87: done: 0.42138 sec","job_status":"done","queue_duration_s":0.187008,"scheduling_latency_s":0.18704,"redis_calls":71,"redis_duration_s":0.14302099999999998,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.013192,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001868,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.126658,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001303,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.171839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00096832,"duration_s":0.42138,"completed_at":"2026-10-04T11:29:59.163Z","load_balancing_strategy":"primary","db_duration_s":0.033809,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 217.326689] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.170Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"49b9417eb18ea3fec765f895","created_at":"2026-10-04T11:29:58.573Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8280d24282c1cb4b884387f61394cb16","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T11:29:58.581Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-49b9417eb18ea3fec765f895: start","job_status":"start","queue_duration_s":0.589052,"scheduling_latency_s":0.589086} vm-test-run-nixbot-gitlab> gitlab # [ 217.341702] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.194Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 217.345154] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.205Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"49b9417eb18ea3fec765f895","created_at":"2026-10-04T11:29:58.573Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8280d24282c1cb4b884387f61394cb16","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T11:29:58.581Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-49b9417eb18ea3fec765f895: done: 0.035295 sec","job_status":"done","queue_duration_s":0.589052,"scheduling_latency_s":0.589086,"redis_calls":10,"redis_duration_s":0.002255,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000435,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001362,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000458,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037548,"duration_s":0.035295,"completed_at":"2026-10-04T11:29:59.205Z","load_balancing_strategy":"primary","db_duration_s":0.000577,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 217.388053] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.207Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"43e5402d473647903b410248","created_at":"2026-10-04T11:29:58.594Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d37949e911b007cdbe5b926f67e8e8a5","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.602Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-43e5402d473647903b410248: start","job_status":"start","queue_duration_s":0.605163,"scheduling_latency_s":0.605188} vm-test-run-nixbot-gitlab> gitlab # [ 217.403392] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.253Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"43e5402d473647903b410248","created_at":"2026-10-04T11:29:58.594Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d37949e911b007cdbe5b926f67e8e8a5","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.602Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-43e5402d473647903b410248: done: 0.045587 sec","job_status":"done","queue_duration_s":0.605163,"scheduling_latency_s":0.605188,"redis_calls":10,"redis_duration_s":0.002079,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000406,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001246,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035116,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.045587,"completed_at":"2026-10-04T11:29:59.252Z","load_balancing_strategy":"primary","db_duration_s":0.014968,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 217.448693] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.254Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"367e11aabc23b7a3a228dbba","created_at":"2026-10-04T11:29:58.636Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"58b561a5417fa2bed09eb5916c866b68","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.648Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-367e11aabc23b7a3a228dbba: start","job_status":"start","queue_duration_s":0.606203,"scheduling_latency_s":0.606225} vm-test-run-nixbot-gitlab> gitlab # [ 217.463627] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.292Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"367e11aabc23b7a3a228dbba","created_at":"2026-10-04T11:29:58.636Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"58b561a5417fa2bed09eb5916c866b68","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.648Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-367e11aabc23b7a3a228dbba: done: 0.037651 sec","job_status":"done","queue_duration_s":0.606203,"scheduling_latency_s":0.606225,"redis_calls":13,"redis_duration_s":0.002792,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000362,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001627,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000803,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025476,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01303368,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.037651,"completed_at":"2026-10-04T11:29:59.292Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003503,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 217.510036] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.294Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"25afa5821570b6a75be9a557","created_at":"2026-10-04T11:29:58.663Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"921ff8e7661c5493f710de2039c095f3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.673Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-25afa5821570b6a75be9a557: start","job_status":"start","queue_duration_s":0.621054,"scheduling_latency_s":0.621072} vm-test-run-nixbot-gitlab> gitlab # [ 217.525299] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.318Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"25afa5821570b6a75be9a557","created_at":"2026-10-04T11:29:58.663Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"921ff8e7661c5493f710de2039c095f3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.673Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-25afa5821570b6a75be9a557: done: 0.024218 sec","job_status":"done","queue_duration_s":0.621054,"scheduling_latency_s":0.621072,"redis_calls":12,"redis_duration_s":0.00227,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000724,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001196,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00035,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027768,"duration_s":0.024218,"completed_at":"2026-10-04T11:29:59.318Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000988,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 217.570133] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.319Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"6c426899b06f7727f0371807","created_at":"2026-10-04T11:29:58.680Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"106d3452be0749489e4a5b63104da020","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T11:29:58.690Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-6c426899b06f7727f0371807: start","job_status":"start","queue_duration_s":0.629007,"scheduling_latency_s":0.629025} vm-test-run-nixbot-gitlab> gitlab # [ 217.584058] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.331Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 217.587757] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.339Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"6c426899b06f7727f0371807","created_at":"2026-10-04T11:29:58.680Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"106d3452be0749489e4a5b63104da020","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T11:29:58.690Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-6c426899b06f7727f0371807: done: 0.020027 sec","job_status":"done","queue_duration_s":0.629007,"scheduling_latency_s":0.629025,"redis_calls":10,"redis_duration_s":0.001703,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000329,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000974,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0004,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01138,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026612,"duration_s":0.020027,"completed_at":"2026-10-04T11:29:59.339Z","load_balancing_strategy":"primary","db_duration_s":0.000478,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 217.631865] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.341Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"72d73897f812bfe6163939a6","created_at":"2026-10-04T11:29:58.703Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"90b4598127ea48bb6162dc4109d39fc4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.722Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-72d73897f812bfe6163939a6: start","job_status":"start","queue_duration_s":0.618678,"scheduling_latency_s":0.618696} vm-test-run-nixbot-gitlab> gitlab # [ 217.647289] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.413Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"72d73897f812bfe6163939a6","created_at":"2026-10-04T11:29:58.703Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"90b4598127ea48bb6162dc4109d39fc4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.722Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-72d73897f812bfe6163939a6: done: 0.072079 sec","job_status":"done","queue_duration_s":0.618678,"scheduling_latency_s":0.618696,"redis_calls":12,"redis_duration_s":0.002317,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000334,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00111,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000873,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006258,"exclusive_lock_hold_duration_s":0.04107338,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.072079,"completed_at":"2026-10-04T11:29:59.413Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016025,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 217.694290] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.415Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"bb2b7fa3204c9ec2e1b7eeb6","created_at":"2026-10-04T11:29:58.738Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ded116abb8ebe7c33b4e72d5b447db6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T11:29:58.766Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-bb2b7fa3204c9ec2e1b7eeb6: start","job_status":"start","queue_duration_s":0.649752,"scheduling_latency_s":0.649779} vm-test-run-nixbot-gitlab> gitlab # [ 217.797058] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.695Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"bb2b7fa3204c9ec2e1b7eeb6","created_at":"2026-10-04T11:29:58.738Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ded116abb8ebe7c33b4e72d5b447db6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T11:29:58.766Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-bb2b7fa3204c9ec2e1b7eeb6: done: 0.279425 sec","job_status":"done","queue_duration_s":0.649752,"scheduling_latency_s":0.649779,"redis_calls":10,"redis_duration_s":0.011446999999999999,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00133,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007579,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002538,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00111012,"duration_s":0.279425,"completed_at":"2026-10-04T11:29:59.695Z","load_balancing_strategy":"primary","db_duration_s":0.009679,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 217.860403] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.702Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"78d2d0b589ce630a405572f3","created_at":"2026-10-04T11:29:58.778Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"869e6542348cb31921fbfade051c5656","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.797Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-78d2d0b589ce630a405572f3: start","job_status":"start","queue_duration_s":0.904267,"scheduling_latency_s":0.904297} vm-test-run-nixbot-gitlab> gitlab # [ 217.915578] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.813Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"78d2d0b589ce630a405572f3","created_at":"2026-10-04T11:29:58.778Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"869e6542348cb31921fbfade051c5656","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.797Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-78d2d0b589ce630a405572f3: done: 0.111601 sec","job_status":"done","queue_duration_s":0.904267,"scheduling_latency_s":0.904297,"redis_calls":19,"redis_duration_s":0.026027,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.005352,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000909,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.014902,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.004864,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042969,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00128186,"duration_s":0.111601,"completed_at":"2026-10-04T11:29:59.813Z","load_balancing_strategy":"primary","db_duration_s":0.011089,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 217.973795] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.826Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a6457fae4afbd18120804db2","created_at":"2026-10-04T11:29:58.809Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0990f9e7eea6af4a71e99fac50ef7bb7","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.824Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a6457fae4afbd18120804db2: start","job_status":"start","queue_duration_s":1.002279,"scheduling_latency_s":1.002318} vm-test-run-nixbot-gitlab> gitlab # [ 218.031172] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.929Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a6457fae4afbd18120804db2","created_at":"2026-10-04T11:29:58.809Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0990f9e7eea6af4a71e99fac50ef7bb7","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.824Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a6457fae4afbd18120804db2: done: 0.102502 sec","job_status":"done","queue_duration_s":1.002279,"scheduling_latency_s":1.002318,"redis_calls":12,"redis_duration_s":0.019576,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003692,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.010437,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005447,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021485,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00124816,"exclusive_lock_hold_duration_s":0.00903632,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.102502,"completed_at":"2026-10-04T11:29:59.929Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008978,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 218.086169] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:29:59.939Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c94a517791f855cb9c9e3a1f","created_at":"2026-10-04T11:29:58.850Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b723d3d0fde839759bbdcd625013869","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.865Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c94a517791f855cb9c9e3a1f: start","job_status":"start","queue_duration_s":1.074131,"scheduling_latency_s":1.074163} vm-test-run-nixbot-gitlab> gitlab # [ 218.114079] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:00.012Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c94a517791f855cb9c9e3a1f","created_at":"2026-10-04T11:29:58.850Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b723d3d0fde839759bbdcd625013869","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.865Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c94a517791f855cb9c9e3a1f: done: 0.072986 sec","job_status":"done","queue_duration_s":1.074131,"scheduling_latency_s":1.074163,"redis_calls":10,"redis_duration_s":0.019159,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004135,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01018,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004844,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00169358,"duration_s":0.072986,"completed_at":"2026-10-04T11:30:00.012Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012251,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 218.162311] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:00.020Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"b9bbd99af4eb80a27bc24716","created_at":"2026-10-04T11:29:58.901Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cba0526b64ac17d90d936820defdc2f2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.914Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-b9bbd99af4eb80a27bc24716: start","job_status":"start","queue_duration_s":1.106338,"scheduling_latency_s":1.106365} vm-test-run-nixbot-gitlab> gitlab # [ 218.193099] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:00.092Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 218.218098] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:00.116Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"b9bbd99af4eb80a27bc24716","created_at":"2026-10-04T11:29:58.901Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cba0526b64ac17d90d936820defdc2f2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.914Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-b9bbd99af4eb80a27bc24716: done: 0.095778 sec","job_status":"done","queue_duration_s":1.106338,"scheduling_latency_s":1.106365,"redis_calls":10,"redis_duration_s":0.015643,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003837,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008889,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002917,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014941,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00258682,"duration_s":0.095778,"completed_at":"2026-10-04T11:30:00.116Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002711,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 218.265239] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:00.124Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"ec821586a721c63cccd4fac3","created_at":"2026-10-04T11:29:58.929Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cddfbf6dab709689563602b889272518","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.940Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-ec821586a721c63cccd4fac3: start","job_status":"start","queue_duration_s":1.184154,"scheduling_latency_s":1.18418} vm-test-run-nixbot-gitlab> gitlab # [ 218.343723] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:00.241Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"ec821586a721c63cccd4fac3","created_at":"2026-10-04T11:29:58.929Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cddfbf6dab709689563602b889272518","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.940Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-ec821586a721c63cccd4fac3: done: 0.117164 sec","job_status":"done","queue_duration_s":1.184154,"scheduling_latency_s":1.18418,"redis_calls":12,"redis_duration_s":0.019935,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004324,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01007,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005541,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00213952,"exclusive_lock_hold_duration_s":0.017098600000000002,"duration_s":0.117164,"completed_at":"2026-10-04T11:30:00.241Z","load_balancing_strategy":"primary","db_duration_s":0.013083,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 218.397058] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:00.253Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"b87bbaee72045de3926437a1","created_at":"2026-10-04T11:29:58.960Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9c522fce8ce11e0687fd2ce6ae7d7773","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.974Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-b87bbaee72045de3926437a1: start","job_status":"start","queue_duration_s":1.279351,"scheduling_latency_s":1.279392} vm-test-run-nixbot-gitlab> gitlab # [ 218.412120] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:00.293Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 218.420603] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:00.318Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"b87bbaee72045de3926437a1","created_at":"2026-10-04T11:29:58.960Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9c522fce8ce11e0687fd2ce6ae7d7773","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:58.974Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-b87bbaee72045de3926437a1: done: 0.065217 sec","job_status":"done","queue_duration_s":1.279351,"scheduling_latency_s":1.279392,"redis_calls":10,"redis_duration_s":0.018038,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001577,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012125,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004336,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00145356,"duration_s":0.065217,"completed_at":"2026-10-04T11:30:00.318Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004479,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 218.464990] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:00.328Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"f0d261257e622f56ae87cc63","created_at":"2026-10-04T11:29:58.995Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f40ae97b1a338d279bd1e9efbd7e0605","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.006Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-f0d261257e622f56ae87cc63: start","job_status":"start","queue_duration_s":1.321116,"scheduling_latency_s":1.321155} vm-test-run-nixbot-gitlab> gitlab # [ 219.092093] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:00.989Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"f0d261257e622f56ae87cc63","created_at":"2026-10-04T11:29:58.995Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f40ae97b1a338d279bd1e9efbd7e0605","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.006Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-f0d261257e622f56ae87cc63: done: 0.661686 sec","job_status":"done","queue_duration_s":1.321116,"scheduling_latency_s":1.321155,"redis_calls":14,"redis_duration_s":0.020848,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002959,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009426,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.008463,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.123,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.123,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.353784,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0035442,"exclusive_lock_hold_duration_s":0.58518704,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.661686,"completed_at":"2026-10-04T11:30:00.989Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.29297,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 219.167611] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:01.065Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"46aae9912b3ce53ffe45656f","created_at":"2026-10-04T11:29:59.019Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17d4c6f7ef93d3779e3c5e53eca7d457","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.030Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-46aae9912b3ce53ffe45656f: start","job_status":"start","queue_duration_s":2.035006,"scheduling_latency_s":2.035046} vm-test-run-nixbot-gitlab> gitlab # [ 219.230018] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:01.128Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"46aae9912b3ce53ffe45656f","created_at":"2026-10-04T11:29:59.019Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17d4c6f7ef93d3779e3c5e53eca7d457","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.030Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-46aae9912b3ce53ffe45656f: done: 0.062296 sec","job_status":"done","queue_duration_s":2.035006,"scheduling_latency_s":2.035046,"redis_calls":10,"redis_duration_s":0.017563,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002719,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009378,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005466,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00111342,"duration_s":0.062296,"completed_at":"2026-10-04T11:30:01.128Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001933,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 219.286417] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:01.139Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"3cd342b37ec7a274d348c8d8","created_at":"2026-10-04T11:29:59.045Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab5b162045b4921b469505aa910e6b7b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.059Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-3cd342b37ec7a274d348c8d8: start","job_status":"start","queue_duration_s":2.079663,"scheduling_latency_s":2.079706} vm-test-run-nixbot-gitlab> gitlab # [ 219.477138] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:01.375Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"3cd342b37ec7a274d348c8d8","created_at":"2026-10-04T11:29:59.045Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab5b162045b4921b469505aa910e6b7b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.059Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-3cd342b37ec7a274d348c8d8: done: 0.235898 sec","job_status":"done","queue_duration_s":2.079663,"scheduling_latency_s":2.079706,"redis_calls":14,"redis_duration_s":0.021061,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.004886,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011029,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005146,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.028,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0017888,"exclusive_lock_hold_duration_s":0.15615421999999998,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.235898,"completed_at":"2026-10-04T11:30:01.375Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.14615,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 219.548393] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:01.384Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"2d1178494adb9b91d151958b","created_at":"2026-10-04T11:29:59.088Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0852aae69974838f8fb8793043bfcbce","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.101Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-2d1178494adb9b91d151958b: start","job_status":"start","queue_duration_s":2.282981,"scheduling_latency_s":2.283018} vm-test-run-nixbot-gitlab> gitlab # [ 219.673749] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:01.572Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 219.695248] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:01.593Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"2d1178494adb9b91d151958b","created_at":"2026-10-04T11:29:59.088Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0852aae69974838f8fb8793043bfcbce","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.101Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-2d1178494adb9b91d151958b: done: 0.209374 sec","job_status":"done","queue_duration_s":2.282981,"scheduling_latency_s":2.283018,"redis_calls":10,"redis_duration_s":0.011063,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003281,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004754,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003028,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031314,"duration_s":0.209374,"completed_at":"2026-10-04T11:30:01.593Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000921,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 219.760407] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:01.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74fbfe5e90fb87cf7f0bf624","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74fbfe5e90fb87cf7f0bf624: start","job_status":"start","queue_duration_s":2.467306,"scheduling_latency_s":2.467335} vm-test-run-nixbot-gitlab> gitlab # [ 219.774953] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:01.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74fbfe5e90fb87cf7f0bf624","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74fbfe5e90fb87cf7f0bf624: done: 0.066191 sec","job_status":"done","queue_duration_s":2.467306,"scheduling_latency_s":2.467335,"redis_calls":16,"redis_duration_s":0.003393,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000549,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000858,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001986,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023258,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035386,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.066191,"completed_at":"2026-10-04T11:30:01.667Z","load_balancing_strategy":"primary","db_duration_s":0.017318,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 219.818177] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:01.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77d1264557dcf605baed305f","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77d1264557dcf605baed305f: start","job_status":"start","queue_duration_s":2.535385,"scheduling_latency_s":2.535412} vm-test-run-nixbot-gitlab> gitlab # [ 219.831238] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:01.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77d1264557dcf605baed305f","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77d1264557dcf605baed305f: done: 0.050968 sec","job_status":"done","queue_duration_s":2.535385,"scheduling_latency_s":2.535412,"redis_calls":13,"redis_duration_s":0.0026880000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001119,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001569,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.050968,"completed_at":"2026-10-04T11:30:01.720Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 219.874355] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:01.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6910bd13165e914df1c894b3","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6910bd13165e914df1c894b3: start","job_status":"start","queue_duration_s":2.588155,"scheduling_latency_s":2.58818} vm-test-run-nixbot-gitlab> gitlab # [ 219.887139] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:01.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6910bd13165e914df1c894b3","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6910bd13165e914df1c894b3: done: 0.037681 sec","job_status":"done","queue_duration_s":2.588155,"scheduling_latency_s":2.58818,"redis_calls":13,"redis_duration_s":0.002512,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000805,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001707,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.037681,"completed_at":"2026-10-04T11:30:01.760Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 219.928683] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:01.762Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aaa2ae82b2282460f78abdfe","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aaa2ae82b2282460f78abdfe: start","job_status":"start","queue_duration_s":2.628485,"scheduling_latency_s":2.628509} vm-test-run-nixbot-gitlab> gitlab # [ 220.078047] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:01.976Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aaa2ae82b2282460f78abdfe","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aaa2ae82b2282460f78abdfe: done: 0.213329 sec","job_status":"done","queue_duration_s":2.628485,"scheduling_latency_s":2.628509,"redis_calls":13,"redis_duration_s":0.05114,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.009947,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.041193,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.213329,"completed_at":"2026-10-04T11:30:01.976Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 220.135956] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:01.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27a192a528f0f27ad5bdd805","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27a192a528f0f27ad5bdd805: start","job_status":"start","queue_duration_s":2.843906,"scheduling_latency_s":2.843936} vm-test-run-nixbot-gitlab> gitlab # [ 220.269817] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:02.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27a192a528f0f27ad5bdd805","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27a192a528f0f27ad5bdd805: done: 0.1899 sec","job_status":"done","queue_duration_s":2.843906,"scheduling_latency_s":2.843936,"redis_calls":13,"redis_duration_s":0.031226,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.012118,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.019108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017154,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.1899,"completed_at":"2026-10-04T11:30:02.168Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 220.329258] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:02.177Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83d615a42f9a0e635bbd74db","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83d615a42f9a0e635bbd74db: start","job_status":"start","queue_duration_s":3.042836,"scheduling_latency_s":3.042862} vm-test-run-nixbot-gitlab> gitlab # [ 220.452628] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:02.350Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83d615a42f9a0e635bbd74db","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83d615a42f9a0e635bbd74db: done: 0.1737 sec","job_status":"done","queue_duration_s":3.042836,"scheduling_latency_s":3.042862,"redis_calls":13,"redis_duration_s":0.025887,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.006768,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.019119,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019281,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.1737,"completed_at":"2026-10-04T11:30:02.350Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 220.553767] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:02.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29296de0c8db8927bc178100","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29296de0c8db8927bc178100: start","job_status":"start","queue_duration_s":3.31798,"scheduling_latency_s":3.318011} vm-test-run-nixbot-gitlab> gitlab # [ 220.734369] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:02.632Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29296de0c8db8927bc178100","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29296de0c8db8927bc178100: done: 0.180346 sec","job_status":"done","queue_duration_s":3.31798,"scheduling_latency_s":3.318011,"redis_calls":13,"redis_duration_s":0.0362,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.008786,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.027414,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.180346,"completed_at":"2026-10-04T11:30:02.632Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 220.792807] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:02.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b69d1abf3915271d1e664e2","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b69d1abf3915271d1e664e2: start","job_status":"start","queue_duration_s":3.512876,"scheduling_latency_s":3.51291} vm-test-run-nixbot-gitlab> gitlab # [ 220.967046] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:02.865Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b69d1abf3915271d1e664e2","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b69d1abf3915271d1e664e2: done: 0.217878 sec","job_status":"done","queue_duration_s":3.512876,"scheduling_latency_s":3.51291,"redis_calls":13,"redis_duration_s":0.042721,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.011253,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.031468,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019769,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.217878,"completed_at":"2026-10-04T11:30:02.865Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 221.023240] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:02.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a691a9ee3898ba88d8c32bde","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a691a9ee3898ba88d8c32bde: start","job_status":"start","queue_duration_s":3.753546,"scheduling_latency_s":3.753581} vm-test-run-nixbot-gitlab> gitlab # [ 221.221102] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.118Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a691a9ee3898ba88d8c32bde","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a691a9ee3898ba88d8c32bde: done: 0.230948 sec","job_status":"done","queue_duration_s":3.753546,"scheduling_latency_s":3.753581,"redis_calls":13,"redis_duration_s":0.040782,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.007431,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.033351,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.230948,"completed_at":"2026-10-04T11:30:03.118Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 221.277325] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdab4a019b9bae7fef205182","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdab4a019b9bae7fef205182: start","job_status":"start","queue_duration_s":4.013656,"scheduling_latency_s":4.01369} vm-test-run-nixbot-gitlab> gitlab # [ 221.435036] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdab4a019b9bae7fef205182","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdab4a019b9bae7fef205182: done: 0.185336 sec","job_status":"done","queue_duration_s":4.013656,"scheduling_latency_s":4.01369,"redis_calls":13,"redis_duration_s":0.019011,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.007229,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.011782,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.185336,"completed_at":"2026-10-04T11:30:03.333Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 221.493363] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.341Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72dc864513b82512a3916e91","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72dc864513b82512a3916e91: start","job_status":"start","queue_duration_s":4.206543,"scheduling_latency_s":4.206577} vm-test-run-nixbot-gitlab> gitlab # [ 221.508048] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.381Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72dc864513b82512a3916e91","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72dc864513b82512a3916e91: done: 0.040662 sec","job_status":"done","queue_duration_s":4.206543,"scheduling_latency_s":4.206577,"redis_calls":13,"redis_duration_s":0.002451,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00153,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.040662,"completed_at":"2026-10-04T11:30:03.381Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 221.552284] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.383Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61967a8b1d7b892c1d63ae53","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61967a8b1d7b892c1d63ae53: start","job_status":"start","queue_duration_s":4.24921,"scheduling_latency_s":4.249242} vm-test-run-nixbot-gitlab> gitlab # [ 221.565207] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.384Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"d458b953098584416dddc367","created_at":"2026-10-04T11:30:03.377Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c5da390b9c7344c1591f905a564fbf0e","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","duplicate-of":"19153d76001d2e9399b96c9f","job_size_bytes":2,"pid":1899,"job_status":"deduplicated","message":"ConcurrencyLimit::ResumeWorker JID-d458b953098584416dddc367: deduplicated: until executing","deduplication.type":"until executing"} vm-test-run-nixbot-gitlab> gitlab # [ 221.578724] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.408Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ab5b00f5bfd6f96aabc80d60","created_at":"2026-10-04T11:30:03.404Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a388366ec0c8ded67a75070d5e921ee1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","duplicate-of":"bdd6d75fb402960301c4e260","job_size_bytes":2,"pid":1899,"job_status":"deduplicated","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ab5b00f5bfd6f96aabc80d60: deduplicated: until executing","deduplication.type":"until executing"} vm-test-run-nixbot-gitlab> gitlab # [ 221.593330] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.424Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61967a8b1d7b892c1d63ae53","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61967a8b1d7b892c1d63ae53: done: 0.041247 sec","job_status":"done","queue_duration_s":4.24921,"scheduling_latency_s":4.249242,"redis_calls":13,"redis_duration_s":0.00719,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006267,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016445,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.041247,"completed_at":"2026-10-04T11:30:03.424Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 221.634402] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.427Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a33b608acd1ff65742acf70","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a33b608acd1ff65742acf70: start","job_status":"start","queue_duration_s":4.292806,"scheduling_latency_s":4.292835} vm-test-run-nixbot-gitlab> gitlab # [ 221.647254] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.447Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a33b608acd1ff65742acf70","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a33b608acd1ff65742acf70: done: 0.020514 sec","job_status":"done","queue_duration_s":4.292806,"scheduling_latency_s":4.292835,"redis_calls":13,"redis_duration_s":0.002078,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000747,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001331,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014049,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020514,"completed_at":"2026-10-04T11:30:03.447Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 221.688691] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"617de849c23b15d50b720776","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-617de849c23b15d50b720776: start","job_status":"start","queue_duration_s":4.314789,"scheduling_latency_s":4.314812} vm-test-run-nixbot-gitlab> gitlab # [ 221.702557] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.466Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"617de849c23b15d50b720776","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-617de849c23b15d50b720776: done: 0.017029 sec","job_status":"done","queue_duration_s":4.314789,"scheduling_latency_s":4.314812,"redis_calls":13,"redis_duration_s":0.0017370000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000622,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001115,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011639,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017029,"completed_at":"2026-10-04T11:30:03.466Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 221.743665] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5bf2e9f14a31114b238043c4","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5bf2e9f14a31114b238043c4: start","job_status":"start","queue_duration_s":4.333312,"scheduling_latency_s":4.333336} vm-test-run-nixbot-gitlab> gitlab # [ 221.756717] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.483Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5bf2e9f14a31114b238043c4","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5bf2e9f14a31114b238043c4: done: 0.016002 sec","job_status":"done","queue_duration_s":4.333312,"scheduling_latency_s":4.333336,"redis_calls":13,"redis_duration_s":0.00157,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010756,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016002,"completed_at":"2026-10-04T11:30:03.483Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 221.797197] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9932a8935befb50b4735c45e","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9932a8935befb50b4735c45e: start","job_status":"start","queue_duration_s":4.350445,"scheduling_latency_s":4.350463} vm-test-run-nixbot-gitlab> gitlab # [ 221.810079] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.499Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9932a8935befb50b4735c45e","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9932a8935befb50b4735c45e: done: 0.014827 sec","job_status":"done","queue_duration_s":4.350445,"scheduling_latency_s":4.350463,"redis_calls":13,"redis_duration_s":0.001686,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001137,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014827,"completed_at":"2026-10-04T11:30:03.499Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 221.850482] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.501Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46a7629e977332e266f17d8a","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46a7629e977332e266f17d8a: start","job_status":"start","queue_duration_s":4.366522,"scheduling_latency_s":4.36654} vm-test-run-nixbot-gitlab> gitlab # [ 221.863285] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.515Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46a7629e977332e266f17d8a","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46a7629e977332e266f17d8a: done: 0.014599 sec","job_status":"done","queue_duration_s":4.366522,"scheduling_latency_s":4.36654,"redis_calls":13,"redis_duration_s":0.001516,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014599,"completed_at":"2026-10-04T11:30:03.515Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 221.905074] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.516Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"203db889e2e00d69b75b53b1","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-203db889e2e00d69b75b53b1: start","job_status":"start","queue_duration_s":4.382403,"scheduling_latency_s":4.382421} vm-test-run-nixbot-gitlab> gitlab # [ 221.917832] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.531Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"203db889e2e00d69b75b53b1","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-203db889e2e00d69b75b53b1: done: 0.014802 sec","job_status":"done","queue_duration_s":4.382403,"scheduling_latency_s":4.382421,"redis_calls":13,"redis_duration_s":0.001521,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014802,"completed_at":"2026-10-04T11:30:03.531Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 221.958612] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.532Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5598e26f15e272e533880cac","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5598e26f15e272e533880cac: start","job_status":"start","queue_duration_s":4.398411,"scheduling_latency_s":4.398429} vm-test-run-nixbot-gitlab> gitlab # [ 221.971800] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.547Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5598e26f15e272e533880cac","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5598e26f15e272e533880cac: done: 0.014989 sec","job_status":"done","queue_duration_s":4.398411,"scheduling_latency_s":4.398429,"redis_calls":13,"redis_duration_s":0.0015769999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014989,"completed_at":"2026-10-04T11:30:03.547Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.012808] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.549Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c409d3c40c30cbbcebc13ac6","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c409d3c40c30cbbcebc13ac6: start","job_status":"start","queue_duration_s":4.414552,"scheduling_latency_s":4.414569} vm-test-run-nixbot-gitlab> gitlab # [ 222.025481] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.563Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c409d3c40c30cbbcebc13ac6","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c409d3c40c30cbbcebc13ac6: done: 0.014147 sec","job_status":"done","queue_duration_s":4.414552,"scheduling_latency_s":4.414569,"redis_calls":13,"redis_duration_s":0.001473,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009694,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014147,"completed_at":"2026-10-04T11:30:03.563Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.065891] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.564Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fee6c838c7cb2941b62faa44","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fee6c838c7cb2941b62faa44: start","job_status":"start","queue_duration_s":4.429975,"scheduling_latency_s":4.429992} vm-test-run-nixbot-gitlab> gitlab # [ 222.078676] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.579Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fee6c838c7cb2941b62faa44","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fee6c838c7cb2941b62faa44: done: 0.015087 sec","job_status":"done","queue_duration_s":4.429975,"scheduling_latency_s":4.429992,"redis_calls":13,"redis_duration_s":0.0016550000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000604,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010274,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015087,"completed_at":"2026-10-04T11:30:03.579Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.120606] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.581Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"141c1c68a6e3aea477e9cdc5","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-141c1c68a6e3aea477e9cdc5: start","job_status":"start","queue_duration_s":4.446538,"scheduling_latency_s":4.446556} vm-test-run-nixbot-gitlab> gitlab # [ 222.133269] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.595Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"141c1c68a6e3aea477e9cdc5","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-141c1c68a6e3aea477e9cdc5: done: 0.014931 sec","job_status":"done","queue_duration_s":4.446538,"scheduling_latency_s":4.446556,"redis_calls":13,"redis_duration_s":0.001638,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010334,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014931,"completed_at":"2026-10-04T11:30:03.595Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.174145] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.597Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63d8d62c6d5e5be1f48ee08e","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63d8d62c6d5e5be1f48ee08e: start","job_status":"start","queue_duration_s":4.463017,"scheduling_latency_s":4.463036} vm-test-run-nixbot-gitlab> gitlab # [ 222.187425] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63d8d62c6d5e5be1f48ee08e","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63d8d62c6d5e5be1f48ee08e: done: 0.014723 sec","job_status":"done","queue_duration_s":4.463017,"scheduling_latency_s":4.463036,"redis_calls":13,"redis_duration_s":0.0015340000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000994,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009865,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014723,"completed_at":"2026-10-04T11:30:03.612Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.228904] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.613Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b9f885e1a4d9cdcb05a4e6e1","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b9f885e1a4d9cdcb05a4e6e1: start","job_status":"start","queue_duration_s":4.47899,"scheduling_latency_s":4.479008} vm-test-run-nixbot-gitlab> gitlab # [ 222.242093] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.628Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b9f885e1a4d9cdcb05a4e6e1","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b9f885e1a4d9cdcb05a4e6e1: done: 0.014703 sec","job_status":"done","queue_duration_s":4.47899,"scheduling_latency_s":4.479008,"redis_calls":13,"redis_duration_s":0.001507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014703,"completed_at":"2026-10-04T11:30:03.628Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.282987] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.629Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b7d20972a6d45b3588c3f51","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b7d20972a6d45b3588c3f51: start","job_status":"start","queue_duration_s":4.49497,"scheduling_latency_s":4.494989} vm-test-run-nixbot-gitlab> gitlab # [ 222.296121] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.643Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b7d20972a6d45b3588c3f51","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b7d20972a6d45b3588c3f51: done: 0.014418 sec","job_status":"done","queue_duration_s":4.49497,"scheduling_latency_s":4.494989,"redis_calls":13,"redis_duration_s":0.0015119999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014418,"completed_at":"2026-10-04T11:30:03.643Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.338185] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.645Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09ace8da73b7e37d26557934","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09ace8da73b7e37d26557934: start","job_status":"start","queue_duration_s":4.510598,"scheduling_latency_s":4.510616} vm-test-run-nixbot-gitlab> gitlab # [ 222.351167] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09ace8da73b7e37d26557934","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09ace8da73b7e37d26557934: done: 0.01436 sec","job_status":"done","queue_duration_s":4.510598,"scheduling_latency_s":4.510616,"redis_calls":13,"redis_duration_s":0.0014680000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01436,"completed_at":"2026-10-04T11:30:03.659Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.392266] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff2f834d565c231cd9d8001e","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff2f834d565c231cd9d8001e: start","job_status":"start","queue_duration_s":4.526247,"scheduling_latency_s":4.526265} vm-test-run-nixbot-gitlab> gitlab # [ 222.405225] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.674Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff2f834d565c231cd9d8001e","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff2f834d565c231cd9d8001e: done: 0.014259 sec","job_status":"done","queue_duration_s":4.526247,"scheduling_latency_s":4.526265,"redis_calls":13,"redis_duration_s":0.001524,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000962,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014259,"completed_at":"2026-10-04T11:30:03.674Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.446098] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.676Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6747aa93fafb7aab8f766131","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6747aa93fafb7aab8f766131: start","job_status":"start","queue_duration_s":4.541726,"scheduling_latency_s":4.541745} vm-test-run-nixbot-gitlab> gitlab # [ 222.459143] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.691Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6747aa93fafb7aab8f766131","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6747aa93fafb7aab8f766131: done: 0.015088 sec","job_status":"done","queue_duration_s":4.541726,"scheduling_latency_s":4.541745,"redis_calls":13,"redis_duration_s":0.001641,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010398,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015088,"completed_at":"2026-10-04T11:30:03.691Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.500235] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.692Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afa7695ab8b1ff5a3915a0e1","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afa7695ab8b1ff5a3915a0e1: start","job_status":"start","queue_duration_s":4.558083,"scheduling_latency_s":4.558101} vm-test-run-nixbot-gitlab> gitlab # [ 222.514261] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.707Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afa7695ab8b1ff5a3915a0e1","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afa7695ab8b1ff5a3915a0e1: done: 0.014612 sec","job_status":"done","queue_duration_s":4.558083,"scheduling_latency_s":4.558101,"redis_calls":13,"redis_duration_s":0.001767,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000561,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001206,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009867,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014612,"completed_at":"2026-10-04T11:30:03.707Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.555356] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58f23edc6e6fd0f230386ea2","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58f23edc6e6fd0f230386ea2: start","job_status":"start","queue_duration_s":4.573956,"scheduling_latency_s":4.573975} vm-test-run-nixbot-gitlab> gitlab # [ 222.568473] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58f23edc6e6fd0f230386ea2","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58f23edc6e6fd0f230386ea2: done: 0.014477 sec","job_status":"done","queue_duration_s":4.573956,"scheduling_latency_s":4.573975,"redis_calls":13,"redis_duration_s":0.001552,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000973,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014477,"completed_at":"2026-10-04T11:30:03.722Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.609532] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.725Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34f9a83583f011ecccf40b6d","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34f9a83583f011ecccf40b6d: start","job_status":"start","queue_duration_s":4.590699,"scheduling_latency_s":4.590718} vm-test-run-nixbot-gitlab> gitlab # [ 222.622452] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.742Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34f9a83583f011ecccf40b6d","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34f9a83583f011ecccf40b6d: done: 0.017262 sec","job_status":"done","queue_duration_s":4.590699,"scheduling_latency_s":4.590718,"redis_calls":13,"redis_duration_s":0.001495,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017262,"completed_at":"2026-10-04T11:30:03.742Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.663592] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"992640586de74e3233c5258e","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-992640586de74e3233c5258e: start","job_status":"start","queue_duration_s":4.609119,"scheduling_latency_s":4.609138} vm-test-run-nixbot-gitlab> gitlab # [ 222.676645] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"992640586de74e3233c5258e","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-992640586de74e3233c5258e: done: 0.01405 sec","job_status":"done","queue_duration_s":4.609119,"scheduling_latency_s":4.609138,"redis_calls":13,"redis_duration_s":0.001459,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01405,"completed_at":"2026-10-04T11:30:03.757Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.718485] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c04653f1ae0f8487dfbd0ad","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c04653f1ae0f8487dfbd0ad: start","job_status":"start","queue_duration_s":4.624336,"scheduling_latency_s":4.624353} vm-test-run-nixbot-gitlab> gitlab # [ 222.731548] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c04653f1ae0f8487dfbd0ad","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c04653f1ae0f8487dfbd0ad: done: 0.014252 sec","job_status":"done","queue_duration_s":4.624336,"scheduling_latency_s":4.624353,"redis_calls":13,"redis_duration_s":0.0015270000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000961,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014252,"completed_at":"2026-10-04T11:30:03.772Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.772875] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc8c332d9c61ce604cbd3e48","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc8c332d9c61ce604cbd3e48: start","job_status":"start","queue_duration_s":4.639762,"scheduling_latency_s":4.63978} vm-test-run-nixbot-gitlab> gitlab # [ 222.785754] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc8c332d9c61ce604cbd3e48","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc8c332d9c61ce604cbd3e48: done: 0.014179 sec","job_status":"done","queue_duration_s":4.639762,"scheduling_latency_s":4.63978,"redis_calls":13,"redis_duration_s":0.001461,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014179,"completed_at":"2026-10-04T11:30:03.788Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.826071] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"950e835b3afa638cfd5d795d","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-950e835b3afa638cfd5d795d: start","job_status":"start","queue_duration_s":4.655103,"scheduling_latency_s":4.65512} vm-test-run-nixbot-gitlab> gitlab # [ 222.838966] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.803Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"950e835b3afa638cfd5d795d","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-950e835b3afa638cfd5d795d: done: 0.014263 sec","job_status":"done","queue_duration_s":4.655103,"scheduling_latency_s":4.65512,"redis_calls":13,"redis_duration_s":0.001662,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001134,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014263,"completed_at":"2026-10-04T11:30:03.803Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.879803] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a755209dc0044ffbf8ef887","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a755209dc0044ffbf8ef887: start","job_status":"start","queue_duration_s":4.670534,"scheduling_latency_s":4.670551} vm-test-run-nixbot-gitlab> gitlab # [ 222.893039] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a755209dc0044ffbf8ef887","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a755209dc0044ffbf8ef887: done: 0.014033 sec","job_status":"done","queue_duration_s":4.670534,"scheduling_latency_s":4.670551,"redis_calls":13,"redis_duration_s":0.0014659999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000926,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014033,"completed_at":"2026-10-04T11:30:03.818Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.935203] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ac817a9363e30298e10a380","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ac817a9363e30298e10a380: start","job_status":"start","queue_duration_s":4.685802,"scheduling_latency_s":4.685821} vm-test-run-nixbot-gitlab> gitlab # [ 222.948314] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.834Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ac817a9363e30298e10a380","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ac817a9363e30298e10a380: done: 0.014159 sec","job_status":"done","queue_duration_s":4.685802,"scheduling_latency_s":4.685821,"redis_calls":13,"redis_duration_s":0.001486,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000959,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009658,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014159,"completed_at":"2026-10-04T11:30:03.834Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.989316] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62f8da5171a87da4f0fab9e8","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62f8da5171a87da4f0fab9e8: start","job_status":"start","queue_duration_s":4.701102,"scheduling_latency_s":4.70112} vm-test-run-nixbot-gitlab> gitlab # [ 223.002101] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62f8da5171a87da4f0fab9e8","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62f8da5171a87da4f0fab9e8: done: 0.014067 sec","job_status":"done","queue_duration_s":4.701102,"scheduling_latency_s":4.70112,"redis_calls":13,"redis_duration_s":0.001494,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000555,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014067,"completed_at":"2026-10-04T11:30:03.849Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.042992] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9aaa96d89dcbfefedb9962ad","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9aaa96d89dcbfefedb9962ad: start","job_status":"start","queue_duration_s":4.716263,"scheduling_latency_s":4.716281} vm-test-run-nixbot-gitlab> gitlab # [ 223.055918] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.864Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9aaa96d89dcbfefedb9962ad","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9aaa96d89dcbfefedb9962ad: done: 0.014083 sec","job_status":"done","queue_duration_s":4.716263,"scheduling_latency_s":4.716281,"redis_calls":13,"redis_duration_s":0.001516,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000567,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014083,"completed_at":"2026-10-04T11:30:03.864Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.096608] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.865Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b95ab5e836b0f7adc570446b","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b95ab5e836b0f7adc570446b: start","job_status":"start","queue_duration_s":4.731441,"scheduling_latency_s":4.731457} vm-test-run-nixbot-gitlab> gitlab # [ 223.109601] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.879Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b95ab5e836b0f7adc570446b","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b95ab5e836b0f7adc570446b: done: 0.01406 sec","job_status":"done","queue_duration_s":4.731441,"scheduling_latency_s":4.731457,"redis_calls":13,"redis_duration_s":0.00151,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00961,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01406,"completed_at":"2026-10-04T11:30:03.879Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.152050] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.881Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b28239345e857c1e19de86d","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b28239345e857c1e19de86d: start","job_status":"start","queue_duration_s":4.746662,"scheduling_latency_s":4.74668} vm-test-run-nixbot-gitlab> gitlab # [ 223.165045] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.902Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b28239345e857c1e19de86d","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b28239345e857c1e19de86d: done: 0.021394 sec","job_status":"done","queue_duration_s":4.746662,"scheduling_latency_s":4.74668,"redis_calls":13,"redis_duration_s":0.003317,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002199,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021394,"completed_at":"2026-10-04T11:30:03.902Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.205390] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a145373bb5cc9e62899a252f","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a145373bb5cc9e62899a252f: start","job_status":"start","queue_duration_s":4.770594,"scheduling_latency_s":4.770613} vm-test-run-nixbot-gitlab> gitlab # [ 223.218472] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a145373bb5cc9e62899a252f","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a145373bb5cc9e62899a252f: done: 0.014143 sec","job_status":"done","queue_duration_s":4.770594,"scheduling_latency_s":4.770613,"redis_calls":13,"redis_duration_s":0.001457,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014143,"completed_at":"2026-10-04T11:30:03.919Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.260363] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4bcd9b8524f338ca66726f9","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4bcd9b8524f338ca66726f9: start","job_status":"start","queue_duration_s":4.785891,"scheduling_latency_s":4.785909} vm-test-run-nixbot-gitlab> gitlab # [ 223.273204] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4bcd9b8524f338ca66726f9","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4bcd9b8524f338ca66726f9: done: 0.014268 sec","job_status":"done","queue_duration_s":4.785891,"scheduling_latency_s":4.785909,"redis_calls":13,"redis_duration_s":0.001523,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014268,"completed_at":"2026-10-04T11:30:03.934Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.314139] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.935Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b468da78f2e15726a8e6e998","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b468da78f2e15726a8e6e998: start","job_status":"start","queue_duration_s":4.80128,"scheduling_latency_s":4.801297} vm-test-run-nixbot-gitlab> gitlab # [ 223.328521] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b468da78f2e15726a8e6e998","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b468da78f2e15726a8e6e998: done: 0.01405 sec","job_status":"done","queue_duration_s":4.80128,"scheduling_latency_s":4.801297,"redis_calls":13,"redis_duration_s":0.001494,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01405,"completed_at":"2026-10-04T11:30:03.949Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.369200] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"824954caf2bd0fd95844830a","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-824954caf2bd0fd95844830a: start","job_status":"start","queue_duration_s":4.816493,"scheduling_latency_s":4.816511} vm-test-run-nixbot-gitlab> gitlab # [ 223.382091] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"824954caf2bd0fd95844830a","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-824954caf2bd0fd95844830a: done: 0.013934 sec","job_status":"done","queue_duration_s":4.816493,"scheduling_latency_s":4.816511,"redis_calls":13,"redis_duration_s":0.0014789999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013934,"completed_at":"2026-10-04T11:30:03.964Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.422891] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f6fdc69259bf05eadcf23b0","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f6fdc69259bf05eadcf23b0: start","job_status":"start","queue_duration_s":4.831535,"scheduling_latency_s":4.831552} vm-test-run-nixbot-gitlab> gitlab # [ 223.435928] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f6fdc69259bf05eadcf23b0","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f6fdc69259bf05eadcf23b0: done: 0.013966 sec","job_status":"done","queue_duration_s":4.831535,"scheduling_latency_s":4.831552,"redis_calls":13,"redis_duration_s":0.001454,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009559,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013966,"completed_at":"2026-10-04T11:30:03.979Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.476861] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e023861b887d13b7c21e57e0","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e023861b887d13b7c21e57e0: start","job_status":"start","queue_duration_s":4.846649,"scheduling_latency_s":4.846667} vm-test-run-nixbot-gitlab> gitlab # [ 223.489742] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e023861b887d13b7c21e57e0","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e023861b887d13b7c21e57e0: done: 0.014258 sec","job_status":"done","queue_duration_s":4.846649,"scheduling_latency_s":4.846667,"redis_calls":13,"redis_duration_s":0.001488,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014258,"completed_at":"2026-10-04T11:30:03.995Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.531331] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:03.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23bc4e08c9b56c3de8175e31","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23bc4e08c9b56c3de8175e31: start","job_status":"start","queue_duration_s":4.862146,"scheduling_latency_s":4.862164} vm-test-run-nixbot-gitlab> gitlab # [ 223.544402] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23bc4e08c9b56c3de8175e31","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23bc4e08c9b56c3de8175e31: done: 0.01433 sec","job_status":"done","queue_duration_s":4.862146,"scheduling_latency_s":4.862164,"redis_calls":13,"redis_duration_s":0.0015439999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00967,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01433,"completed_at":"2026-10-04T11:30:04.010Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.585179] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63af72a65536431545081fe4","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63af72a65536431545081fe4: start","job_status":"start","queue_duration_s":4.877635,"scheduling_latency_s":4.877653} vm-test-run-nixbot-gitlab> gitlab # [ 223.598140] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.026Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63af72a65536431545081fe4","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63af72a65536431545081fe4: done: 0.013967 sec","job_status":"done","queue_duration_s":4.877635,"scheduling_latency_s":4.877653,"redis_calls":13,"redis_duration_s":0.001486,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000959,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013967,"completed_at":"2026-10-04T11:30:04.025Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.639293] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.027Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8357cd0272fd65015247c1c6","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8357cd0272fd65015247c1c6: start","job_status":"start","queue_duration_s":4.892732,"scheduling_latency_s":4.89275} vm-test-run-nixbot-gitlab> gitlab # [ 223.652212] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.041Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8357cd0272fd65015247c1c6","created_at":"2026-10-04T11:29:58.816Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-fab267840a294c0a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b786d6e7b4e74099ecb4cbf8b0f8ebae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8357cd0272fd65015247c1c6: done: 0.014131 sec","job_status":"done","queue_duration_s":4.892732,"scheduling_latency_s":4.89275,"redis_calls":13,"redis_duration_s":0.001582,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014131,"completed_at":"2026-10-04T11:30:04.041Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.692739] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.042Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"19153d76001d2e9399b96c9f","created_at":"2026-10-04T11:29:59.125Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e8be07c8f2a2856ae485537f6c558a0","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.140Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-19153d76001d2e9399b96c9f: start","job_status":"start","queue_duration_s":4.901495,"scheduling_latency_s":4.901514} vm-test-run-nixbot-gitlab> gitlab # [ 223.707563] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.355Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"19153d76001d2e9399b96c9f","created_at":"2026-10-04T11:29:59.125Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e8be07c8f2a2856ae485537f6c558a0","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.140Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-19153d76001d2e9399b96c9f: done: 0.312898 sec","job_status":"done","queue_duration_s":4.901495,"scheduling_latency_s":4.901514,"redis_calls":565,"redis_duration_s":0.058329,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000308,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000765,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.057256,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.287573,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025982,"duration_s":0.312898,"completed_at":"2026-10-04T11:30:04.355Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000614,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.753246] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.356Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"bdd6d75fb402960301c4e260","created_at":"2026-10-04T11:29:59.156Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"25a247b57bb0db6360b2bdeca918a2db","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.167Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-bdd6d75fb402960301c4e260: start","job_status":"start","queue_duration_s":5.189722,"scheduling_latency_s":5.18974} vm-test-run-nixbot-gitlab> gitlab # [ 223.768449] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.368Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 223.772318] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.376Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"bdd6d75fb402960301c4e260","created_at":"2026-10-04T11:29:59.156Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"25a247b57bb0db6360b2bdeca918a2db","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.167Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-bdd6d75fb402960301c4e260: done: 0.01941 sec","job_status":"done","queue_duration_s":5.189722,"scheduling_latency_s":5.18974,"redis_calls":10,"redis_duration_s":0.001437,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000317,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000774,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000346,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011863,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002729,"duration_s":0.01941,"completed_at":"2026-10-04T11:30:04.376Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000575,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.816602] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.377Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6c2bc9310995ddb3e4638632","created_at":"2026-10-04T11:29:59.745Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a2dd09ee46684ffe","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"869e6542348cb31921fbfade051c5656","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.786Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6c2bc9310995ddb3e4638632: start","job_status":"start","queue_duration_s":4.590617,"scheduling_latency_s":4.590636} vm-test-run-nixbot-gitlab> gitlab # [ 223.829310] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.395Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 223.830996] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.409Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6c2bc9310995ddb3e4638632","created_at":"2026-10-04T11:29:59.745Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a2dd09ee46684ffe","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"869e6542348cb31921fbfade051c5656","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.786Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6c2bc9310995ddb3e4638632: done: 0.031625 sec","job_status":"done","queue_duration_s":4.590617,"scheduling_latency_s":4.590636,"redis_calls":13,"redis_duration_s":0.001888,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000312,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000724,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000852,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021525,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025458,"duration_s":0.031625,"completed_at":"2026-10-04T11:30:04.409Z","load_balancing_strategy":"primary","db_duration_s":0.003568,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.872742] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.410Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ec7204545df304d97ce3ccf1","created_at":"2026-10-04T11:29:59.745Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a2dd09ee46684ffe","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"869e6542348cb31921fbfade051c5656","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.786Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ec7204545df304d97ce3ccf1: start","job_status":"start","queue_duration_s":4.623393,"scheduling_latency_s":4.623409} vm-test-run-nixbot-gitlab> gitlab # [ 223.885335] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.420Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 223.887086] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.429Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ec7204545df304d97ce3ccf1","created_at":"2026-10-04T11:29:59.745Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a2dd09ee46684ffe","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"869e6542348cb31921fbfade051c5656","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.786Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ec7204545df304d97ce3ccf1: done: 0.018872 sec","job_status":"done","queue_duration_s":4.623393,"scheduling_latency_s":4.623409,"redis_calls":10,"redis_duration_s":0.001402,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000782,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013545,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018872,"completed_at":"2026-10-04T11:30:04.429Z","load_balancing_strategy":"primary","db_duration_s":0.001302,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.927178] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.430Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f3a8814c976cfc88b1001c7d","created_at":"2026-10-04T11:29:59.745Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a2dd09ee46684ffe","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"869e6542348cb31921fbfade051c5656","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.786Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f3a8814c976cfc88b1001c7d: start","job_status":"start","queue_duration_s":4.643392,"scheduling_latency_s":4.643411} vm-test-run-nixbot-gitlab> gitlab # [ 223.941095] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.440Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 223.942825] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.449Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f3a8814c976cfc88b1001c7d","created_at":"2026-10-04T11:29:59.745Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a2dd09ee46684ffe","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"869e6542348cb31921fbfade051c5656","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.786Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f3a8814c976cfc88b1001c7d: done: 0.018937 sec","job_status":"done","queue_duration_s":4.643392,"scheduling_latency_s":4.643411,"redis_calls":10,"redis_duration_s":0.0016640000000000001,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000864,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.0008,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018937,"completed_at":"2026-10-04T11:30:04.449Z","load_balancing_strategy":"primary","db_duration_s":0.001363,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.983366] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.450Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"260166c65d2e002d3a0faf50","created_at":"2026-10-04T11:29:59.745Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a2dd09ee46684ffe","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"869e6542348cb31921fbfade051c5656","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.786Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-260166c65d2e002d3a0faf50: start","job_status":"start","queue_duration_s":4.663448,"scheduling_latency_s":4.663465} vm-test-run-nixbot-gitlab> gitlab # [ 223.996116] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.460Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 223.997855] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.469Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"260166c65d2e002d3a0faf50","created_at":"2026-10-04T11:29:59.745Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a2dd09ee46684ffe","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"869e6542348cb31921fbfade051c5656","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:29:59.786Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-260166c65d2e002d3a0faf50: done: 0.01864 sec","job_status":"done","queue_duration_s":4.663448,"scheduling_latency_s":4.663465,"redis_calls":10,"redis_duration_s":0.00141,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000612,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000798,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013435,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01864,"completed_at":"2026-10-04T11:30:04.468Z","load_balancing_strategy":"primary","db_duration_s":0.001474,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.037864] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.470Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"ab8eabad9db58096fd2ab3ee","created_at":"2026-10-04T11:30:01.798Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ed831055b1cbde18ff25b00cb3cc3ab","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:01.833Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-ab8eabad9db58096fd2ab3ee: start","job_status":"start","queue_duration_s":2.636908,"scheduling_latency_s":2.636926} vm-test-run-nixbot-gitlab> gitlab # [ 224.052582] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.563Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"ab8eabad9db58096fd2ab3ee","created_at":"2026-10-04T11:30:01.798Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ed831055b1cbde18ff25b00cb3cc3ab","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:01.833Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-ab8eabad9db58096fd2ab3ee: done: 0.093651 sec","job_status":"done","queue_duration_s":2.636908,"scheduling_latency_s":2.636926,"redis_calls":11,"redis_duration_s":0.001707,"redis_read_bytes":9,"redis_write_bytes":1239,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000286,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000987,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000434,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":298,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024252,"duration_s":0.093651,"completed_at":"2026-10-04T11:30:04.563Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000327,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.096783] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.565Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"ffe58f42e2b015b832e54361","created_at":"2026-10-04T11:30:01.855Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"793ca31b9e59be3ebdfc4f4c1c3b88b4","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:01.861Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-ffe58f42e2b015b832e54361: start","job_status":"start","queue_duration_s":2.70342,"scheduling_latency_s":2.703438} vm-test-run-nixbot-gitlab> gitlab # [ 224.111660] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.584Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"ffe58f42e2b015b832e54361","created_at":"2026-10-04T11:30:01.855Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"793ca31b9e59be3ebdfc4f4c1c3b88b4","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:01.861Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-ffe58f42e2b015b832e54361: done: 0.019062 sec","job_status":"done","queue_duration_s":2.70342,"scheduling_latency_s":2.703438,"redis_calls":7,"redis_duration_s":0.001046,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000761,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000285,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019062,"completed_at":"2026-10-04T11:30:04.584Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00319,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.155157] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.585Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"1f551891f7d1cf31b9d9752e","created_at":"2026-10-04T11:30:01.895Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"86a240d63e36d3152bac6bc275310390","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:01.925Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-1f551891f7d1cf31b9d9752e: start","job_status":"start","queue_duration_s":2.660007,"scheduling_latency_s":2.660026} vm-test-run-nixbot-gitlab> gitlab # [ 224.168782] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.613Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"1f551891f7d1cf31b9d9752e","created_at":"2026-10-04T11:30:01.895Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"86a240d63e36d3152bac6bc275310390","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:01.925Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-1f551891f7d1cf31b9d9752e: done: 0.02752 sec","job_status":"done","queue_duration_s":2.660007,"scheduling_latency_s":2.660026,"redis_calls":11,"redis_duration_s":0.0015890000000000001,"redis_read_bytes":10,"redis_write_bytes":1022,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000297,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":283,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000839,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000453,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":225,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015863,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002569,"duration_s":0.02752,"completed_at":"2026-10-04T11:30:04.613Z","load_balancing_strategy":"primary","db_duration_s":0.003893,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.211398] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.614Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Import::ImportFileCleanupWorker","jid":"a8ce34573833b946ea0fcbc5","created_at":"2026-10-04T11:30:01.951Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d15442cd8a7d481ebdde040af8102a5","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:872203d04eabf30611038084c617b8ff07fd3e5607b1d8e2c977d615a3cfd3cb","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.114Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Import::ImportFileCleanupWorker JID-a8ce34573833b946ea0fcbc5: start","job_status":"start","queue_duration_s":2.50006,"scheduling_latency_s":2.500079} vm-test-run-nixbot-gitlab> gitlab # [ 224.226569] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.648Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Import::ImportFileCleanupWorker","jid":"a8ce34573833b946ea0fcbc5","created_at":"2026-10-04T11:30:01.951Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d15442cd8a7d481ebdde040af8102a5","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:872203d04eabf30611038084c617b8ff07fd3e5607b1d8e2c977d615a3cfd3cb","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.114Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Import::ImportFileCleanupWorker JID-a8ce34573833b946ea0fcbc5: done: 0.034277 sec","job_status":"done","queue_duration_s":2.50006,"scheduling_latency_s":2.500079,"redis_calls":10,"redis_duration_s":0.001951,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00034,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00127,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000341,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01742,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024684,"duration_s":0.034277,"completed_at":"2026-10-04T11:30:04.648Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008771,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.270993] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.649Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"e19e31a6484b5f3a56839152","created_at":"2026-10-04T11:30:02.135Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.142Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-e19e31a6484b5f3a56839152: start","job_status":"start","queue_duration_s":2.507237,"scheduling_latency_s":2.507255} vm-test-run-nixbot-gitlab> gitlab # [ 224.285277] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.745Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"e19e31a6484b5f3a56839152","created_at":"2026-10-04T11:30:02.135Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.142Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-e19e31a6484b5f3a56839152: done: 0.09581 sec","job_status":"done","queue_duration_s":2.507237,"scheduling_latency_s":2.507255,"redis_calls":60,"redis_duration_s":0.009167,"redis_read_bytes":108,"redis_write_bytes":41059,"redis_queues_calls":2,"redis_queues_duration_s":0.000696,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.008066,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000405,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.088133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.09581,"completed_at":"2026-10-04T11:30:04.745Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.327956] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.746Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"15dae9ef5f6b88ef773c4666","created_at":"2026-10-04T11:30:02.159Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f002782721a93190b8a954b56a81948","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T11:30:02.166Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-15dae9ef5f6b88ef773c4666: start","job_status":"start","queue_duration_s":2.580097,"scheduling_latency_s":2.580113} vm-test-run-nixbot-gitlab> gitlab # [ 224.342134] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.754Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 224.346655] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.759Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"15dae9ef5f6b88ef773c4666","created_at":"2026-10-04T11:30:02.159Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f002782721a93190b8a954b56a81948","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T11:30:02.166Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-15dae9ef5f6b88ef773c4666: done: 0.013205 sec","job_status":"done","queue_duration_s":2.580097,"scheduling_latency_s":2.580113,"redis_calls":7,"redis_duration_s":0.001019,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000717,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000302,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008735,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013205,"completed_at":"2026-10-04T11:30:04.759Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.387723] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.761Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"64c4e4fc338e78bc7346bc67","created_at":"2026-10-04T11:30:02.176Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0629fd7ff1261e1f7cfa307c4dc9828e","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.196Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-64c4e4fc338e78bc7346bc67: start","job_status":"start","queue_duration_s":2.564637,"scheduling_latency_s":2.564655} vm-test-run-nixbot-gitlab> gitlab # [ 224.402494] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.782Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"64c4e4fc338e78bc7346bc67","created_at":"2026-10-04T11:30:02.176Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0629fd7ff1261e1f7cfa307c4dc9828e","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.196Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-64c4e4fc338e78bc7346bc67: done: 0.021195 sec","job_status":"done","queue_duration_s":2.564637,"scheduling_latency_s":2.564655,"redis_calls":11,"redis_duration_s":0.001662,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000287,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000805,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000258,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000312,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024022,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.021195,"completed_at":"2026-10-04T11:30:04.782Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000604,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.448508] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.783Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"9a997dee5fdec9c0f31430b6","created_at":"2026-10-04T11:30:02.200Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"03985618adf5fe0b79eb3aebbce64795","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.217Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-9a997dee5fdec9c0f31430b6: start","job_status":"start","queue_duration_s":2.565707,"scheduling_latency_s":2.565724} vm-test-run-nixbot-gitlab> gitlab # [ 224.463328] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.804Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"9a997dee5fdec9c0f31430b6","created_at":"2026-10-04T11:30:02.200Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"03985618adf5fe0b79eb3aebbce64795","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.217Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-9a997dee5fdec9c0f31430b6: done: 0.020711 sec","job_status":"done","queue_duration_s":2.565707,"scheduling_latency_s":2.565724,"redis_calls":12,"redis_duration_s":0.0016279999999999999,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000283,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000753,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000592,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023566,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.020711,"completed_at":"2026-10-04T11:30:04.804Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000417,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.510894] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.805Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Export::PruneProjectExportJobsWorker","jid":"f1f340514e9a8f565e3418d1","created_at":"2026-10-04T11:30:02.224Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce70f82bf38ec234f97cd49e6cd02839","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b2f7045536fe38235224f4d98e515e25b0b0eea1aa1ae4b4ab9f99ab22c53b43","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.242Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Export::PruneProjectExportJobsWorker JID-f1f340514e9a8f565e3418d1: start","job_status":"start","queue_duration_s":2.562636,"scheduling_latency_s":2.562654} vm-test-run-nixbot-gitlab> gitlab # [ 224.526182] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.835Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Export::PruneProjectExportJobsWorker","jid":"f1f340514e9a8f565e3418d1","created_at":"2026-10-04T11:30:02.224Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce70f82bf38ec234f97cd49e6cd02839","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b2f7045536fe38235224f4d98e515e25b0b0eea1aa1ae4b4ab9f99ab22c53b43","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.242Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Export::PruneProjectExportJobsWorker JID-f1f340514e9a8f565e3418d1: done: 0.030082 sec","job_status":"done","queue_duration_s":2.562636,"scheduling_latency_s":2.562654,"redis_calls":10,"redis_duration_s":0.001386,"redis_read_bytes":9,"redis_write_bytes":1173,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00029,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000796,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0003,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":236,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025062,"duration_s":0.030082,"completed_at":"2026-10-04T11:30:04.835Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004458,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.570873] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.836Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"d14eab1937aac5960389041d","created_at":"2026-10-04T11:30:02.257Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"678b1caaf8a14d36e7ae93459f80d4a6","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.268Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-d14eab1937aac5960389041d: start","job_status":"start","queue_duration_s":2.567626,"scheduling_latency_s":2.567642} vm-test-run-nixbot-gitlab> gitlab # [ 224.585450] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.858Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"d14eab1937aac5960389041d","created_at":"2026-10-04T11:30:02.257Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"678b1caaf8a14d36e7ae93459f80d4a6","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.268Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-d14eab1937aac5960389041d: done: 0.021625 sec","job_status":"done","queue_duration_s":2.567626,"scheduling_latency_s":2.567642,"redis_calls":7,"redis_duration_s":0.00098,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000704,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000276,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.021625,"completed_at":"2026-10-04T11:30:04.858Z","load_balancing_strategy":"primary","db_duration_s":0.008246,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.627767] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.860Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"51b89fd6431a373a38aae94f","created_at":"2026-10-04T11:30:02.290Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"836766151ef179734aa9a84866c98a32","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.313Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-51b89fd6431a373a38aae94f: start","job_status":"start","queue_duration_s":2.546406,"scheduling_latency_s":2.546424} vm-test-run-nixbot-gitlab> gitlab # [ 224.643008] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.888Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"51b89fd6431a373a38aae94f","created_at":"2026-10-04T11:30:02.290Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"836766151ef179734aa9a84866c98a32","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.313Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-51b89fd6431a373a38aae94f: done: 0.027778 sec","job_status":"done","queue_duration_s":2.546406,"scheduling_latency_s":2.546424,"redis_calls":10,"redis_duration_s":0.001386,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000283,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000779,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000324,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017219,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025032,"duration_s":0.027778,"completed_at":"2026-10-04T11:30:04.888Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003535,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.686862] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.889Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckExportJobsWorker","jid":"f01d04c71b23fa5e9b7e72d9","created_at":"2026-10-04T11:30:02.337Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1b1f4e837ea8dc93de51e5d48e96def","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:c05136a1912f23c3950f9f6243170d655fcc26d1e5b96896d7e248d72f9039fe","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.388Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckExportJobsWorker JID-f01d04c71b23fa5e9b7e72d9: start","job_status":"start","queue_duration_s":2.500393,"scheduling_latency_s":2.500411} vm-test-run-nixbot-gitlab> gitlab # [ 224.700629] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.910Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckExportJobsWorker","jid":"f01d04c71b23fa5e9b7e72d9","created_at":"2026-10-04T11:30:02.337Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1b1f4e837ea8dc93de51e5d48e96def","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:c05136a1912f23c3950f9f6243170d655fcc26d1e5b96896d7e248d72f9039fe","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.388Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckExportJobsWorker JID-f01d04c71b23fa5e9b7e72d9: done: 0.021135 sec","job_status":"done","queue_duration_s":2.500393,"scheduling_latency_s":2.500411,"redis_calls":10,"redis_duration_s":0.001376,"redis_read_bytes":9,"redis_write_bytes":995,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000292,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000796,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":518,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000288,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":192,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023798,"duration_s":0.021135,"completed_at":"2026-10-04T11:30:04.910Z","load_balancing_strategy":"primary","db_duration_s":0.000958,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.742688] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.911Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"eebbd3a775e8a2b17c9eab59","created_at":"2026-10-04T11:30:02.465Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"36654e8740b4e74a3b9196cff1329b9d","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.484Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-eebbd3a775e8a2b17c9eab59: start","job_status":"start","queue_duration_s":2.4274,"scheduling_latency_s":2.427417} vm-test-run-nixbot-gitlab> gitlab # [ 224.757449] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.933Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"eebbd3a775e8a2b17c9eab59","created_at":"2026-10-04T11:30:02.465Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"36654e8740b4e74a3b9196cff1329b9d","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.484Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-eebbd3a775e8a2b17c9eab59: done: 0.022183 sec","job_status":"done","queue_duration_s":2.4274,"scheduling_latency_s":2.427417,"redis_calls":12,"redis_duration_s":0.00199,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000657,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000281,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000736,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000316,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011624,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026012,"duration_s":0.022183,"completed_at":"2026-10-04T11:30:04.933Z","load_balancing_strategy":"primary","db_duration_s":0.001921,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.801920] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.934Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"539e7f9408704b8de6f0de73","created_at":"2026-10-04T11:30:02.502Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53f9bffdd6527bfe41111fc40e5c8253","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.522Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-539e7f9408704b8de6f0de73: start","job_status":"start","queue_duration_s":2.412549,"scheduling_latency_s":2.412567} vm-test-run-nixbot-gitlab> gitlab # [ 224.817192] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.967Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"539e7f9408704b8de6f0de73","created_at":"2026-10-04T11:30:02.502Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53f9bffdd6527bfe41111fc40e5c8253","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.522Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-539e7f9408704b8de6f0de73: done: 0.032479 sec","job_status":"done","queue_duration_s":2.412549,"scheduling_latency_s":2.412567,"redis_calls":20,"redis_duration_s":0.002748,"redis_read_bytes":23,"redis_write_bytes":4317,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.000849,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1065,"redis_queues_calls":4,"redis_queues_duration_s":0.000545,"redis_queues_read_bytes":6,"redis_queues_write_bytes":2022,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001084,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":1004,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00027,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024934,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027406,"duration_s":0.032479,"completed_at":"2026-10-04T11:30:04.967Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001349,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.862608] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.969Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"f6efde11a346d2b777265b9a","created_at":"2026-10-04T11:30:02.551Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0abc87a299da41480c8bdb5173fe1948","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.560Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-f6efde11a346d2b777265b9a: start","job_status":"start","queue_duration_s":2.4086,"scheduling_latency_s":2.408619} vm-test-run-nixbot-gitlab> gitlab # [ 224.878075] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.990Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"f6efde11a346d2b777265b9a","created_at":"2026-10-04T11:30:02.551Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0abc87a299da41480c8bdb5173fe1948","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.560Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-f6efde11a346d2b777265b9a: done: 0.020567 sec","job_status":"done","queue_duration_s":2.4086,"scheduling_latency_s":2.408619,"redis_calls":10,"redis_duration_s":0.0015,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000623,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00510544,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.020567,"completed_at":"2026-10-04T11:30:04.989Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002339,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 224.922081] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:04.991Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"26f21b703620a4a497c23eed","created_at":"2026-10-04T11:30:02.568Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13ad4bee087e8ef4d3a8ef46ad710375","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.596Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-26f21b703620a4a497c23eed: start","job_status":"start","queue_duration_s":2.394435,"scheduling_latency_s":2.394452} vm-test-run-nixbot-gitlab> gitlab # [ 224.936162] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.015Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"26f21b703620a4a497c23eed","created_at":"2026-10-04T11:30:02.568Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13ad4bee087e8ef4d3a8ef46ad710375","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.596Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-26f21b703620a4a497c23eed: done: 0.023973 sec","job_status":"done","queue_duration_s":2.394435,"scheduling_latency_s":2.394452,"redis_calls":10,"redis_duration_s":0.001399,"redis_read_bytes":9,"redis_write_bytes":1179,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000302,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000812,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000285,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":238,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016138,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002451,"duration_s":0.023973,"completed_at":"2026-10-04T11:30:05.015Z","load_balancing_strategy":"primary","db_duration_s":0.000401,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.980685] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.016Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"073ab4e5c67afdc6be31bf02","created_at":"2026-10-04T11:30:02.618Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd8bbecd5ef6b75e1daa616dc1fc5d4f","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.641Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-073ab4e5c67afdc6be31bf02: start","job_status":"start","queue_duration_s":2.374352,"scheduling_latency_s":2.374369} vm-test-run-nixbot-gitlab> gitlab # [ 225.002415] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.044Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"073ab4e5c67afdc6be31bf02","created_at":"2026-10-04T11:30:02.618Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd8bbecd5ef6b75e1daa616dc1fc5d4f","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.641Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-073ab4e5c67afdc6be31bf02: done: 0.028129 sec","job_status":"done","queue_duration_s":2.374352,"scheduling_latency_s":2.374369,"redis_calls":10,"redis_duration_s":0.001322,"redis_read_bytes":9,"redis_write_bytes":1097,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000268,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":309,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000771,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000283,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":218,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024334,"duration_s":0.028129,"completed_at":"2026-10-04T11:30:05.044Z","load_balancing_strategy":"primary","db_duration_s":0.004321,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.057167] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.045Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","jid":"99893a4fa703d997a13e728d","created_at":"2026-10-04T11:30:02.646Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b20d2b0eadfc3af417057246635680a9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fd978bae1387dafc89025ea66dbcf41624e32b5e06562317011e023b1327155","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.715Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleBulkDeleteJobArtifactCronWorker JID-99893a4fa703d997a13e728d: start","job_status":"start","queue_duration_s":2.330483,"scheduling_latency_s":2.330499} vm-test-run-nixbot-gitlab> gitlab # [ 225.072434] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.083Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","jid":"99893a4fa703d997a13e728d","created_at":"2026-10-04T11:30:02.646Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b20d2b0eadfc3af417057246635680a9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fd978bae1387dafc89025ea66dbcf41624e32b5e06562317011e023b1327155","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.715Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleBulkDeleteJobArtifactCronWorker JID-99893a4fa703d997a13e728d: done: 0.038485 sec","job_status":"done","queue_duration_s":2.330483,"scheduling_latency_s":2.330499,"redis_calls":24,"redis_duration_s":0.003242,"redis_read_bytes":25,"redis_write_bytes":5977,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000625,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":704,"redis_queues_calls":2,"redis_queues_duration_s":0.000226,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3810,"redis_queues_metadata_calls":11,"redis_queues_metadata_duration_s":0.001554,"redis_queues_metadata_read_bytes":14,"redis_queues_metadata_write_bytes":894,"redis_shared_state_calls":7,"redis_shared_state_duration_s":0.000837,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":569,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027492,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002465,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.max_buckets":5,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.available_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.occupied_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.stale_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.missing_count":5,"duration_s":0.038485,"completed_at":"2026-10-04T11:30:05.083Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000985,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.122332] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.085Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"1972c684b80e9ac7109334e4","created_at":"2026-10-04T11:30:02.729Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5bc91331a1527a3a6536b06161e65f1b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-1972c684b80e9ac7109334e4: start","job_status":"start","queue_duration_s":2.34792,"scheduling_latency_s":2.347936} vm-test-run-nixbot-gitlab> gitlab # [ 225.137893] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.097Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"1972c684b80e9ac7109334e4","created_at":"2026-10-04T11:30:02.729Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5bc91331a1527a3a6536b06161e65f1b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-1972c684b80e9ac7109334e4: done: 0.012631 sec","job_status":"done","queue_duration_s":2.34792,"scheduling_latency_s":2.347936,"redis_calls":7,"redis_duration_s":0.000965,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000693,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008288,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012631,"completed_at":"2026-10-04T11:30:05.097Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.182328] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.098Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"1bce948138714d5baebae250","created_at":"2026-10-04T11:30:02.750Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"643db41578d2a4d078e756a14f9653de","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T11:30:02.758Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-1bce948138714d5baebae250: start","job_status":"start","queue_duration_s":2.340778,"scheduling_latency_s":2.340795} vm-test-run-nixbot-gitlab> gitlab # [ 225.196203] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.105Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 225.200126] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.112Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"1bce948138714d5baebae250","created_at":"2026-10-04T11:30:02.750Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"643db41578d2a4d078e756a14f9653de","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T11:30:02.758Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-1bce948138714d5baebae250: done: 0.013423 sec","job_status":"done","queue_duration_s":2.340778,"scheduling_latency_s":2.340795,"redis_calls":7,"redis_duration_s":0.000988,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000718,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00027,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013423,"completed_at":"2026-10-04T11:30:05.112Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.242011] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.113Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"942a1e9ef50d4a9d28977dee","created_at":"2026-10-04T11:30:02.781Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93328725ede77c2613540a834d93e65a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.789Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-942a1e9ef50d4a9d28977dee: start","job_status":"start","queue_duration_s":2.324085,"scheduling_latency_s":2.324102} vm-test-run-nixbot-gitlab> gitlab # [ 225.257328] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.148Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"942a1e9ef50d4a9d28977dee","created_at":"2026-10-04T11:30:02.781Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93328725ede77c2613540a834d93e65a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.789Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-942a1e9ef50d4a9d28977dee: done: 0.035041 sec","job_status":"done","queue_duration_s":2.324085,"scheduling_latency_s":2.324102,"redis_calls":9,"redis_duration_s":0.001367,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000713,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000654,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00055048,"exclusive_lock_hold_duration_s":0.01423854,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.035041,"completed_at":"2026-10-04T11:30:05.148Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013193,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.302959] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.149Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b6572622bd453285490b6fe8","created_at":"2026-10-04T11:30:02.813Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bdd5561d5344c08135caea3f7dbcde42","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T11:30:02.823Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b6572622bd453285490b6fe8: start","job_status":"start","queue_duration_s":2.326371,"scheduling_latency_s":2.326388} vm-test-run-nixbot-gitlab> gitlab # [ 225.317084] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.170Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b6572622bd453285490b6fe8","created_at":"2026-10-04T11:30:02.813Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bdd5561d5344c08135caea3f7dbcde42","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T11:30:02.823Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b6572622bd453285490b6fe8: done: 0.020586 sec","job_status":"done","queue_duration_s":2.326371,"scheduling_latency_s":2.326388,"redis_calls":7,"redis_duration_s":0.0010350000000000001,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00076,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000275,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014747,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020586,"completed_at":"2026-10-04T11:30:05.170Z","load_balancing_strategy":"primary","db_duration_s":0.000878,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.358028] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.171Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"7eaaf8bc5385bfecf3ab4434","created_at":"2026-10-04T11:30:02.842Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8f83b6976632dd2f964b9d48404bd010","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.849Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-7eaaf8bc5385bfecf3ab4434: start","job_status":"start","queue_duration_s":2.322029,"scheduling_latency_s":2.322047} vm-test-run-nixbot-gitlab> gitlab # [ 225.373231] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.197Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"7eaaf8bc5385bfecf3ab4434","created_at":"2026-10-04T11:30:02.842Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8f83b6976632dd2f964b9d48404bd010","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.849Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-7eaaf8bc5385bfecf3ab4434: done: 0.025636 sec","job_status":"done","queue_duration_s":2.322029,"scheduling_latency_s":2.322047,"redis_calls":14,"redis_duration_s":0.002003,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000181,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001376,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000446,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025636,"completed_at":"2026-10-04T11:30:05.197Z","load_balancing_strategy":"primary","db_duration_s":0.00057,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.415822] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.198Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d29622821d4467332a0eee40","created_at":"2026-10-04T11:30:02.866Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"24b0d7ad3902802d8283ceca5ab57cff","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d29622821d4467332a0eee40: start","job_status":"start","queue_duration_s":2.326599,"scheduling_latency_s":2.326616} vm-test-run-nixbot-gitlab> gitlab # [ 225.431020] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.214Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d29622821d4467332a0eee40","created_at":"2026-10-04T11:30:02.866Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"24b0d7ad3902802d8283ceca5ab57cff","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d29622821d4467332a0eee40: done: 0.015838 sec","job_status":"done","queue_duration_s":2.326599,"scheduling_latency_s":2.326616,"redis_calls":9,"redis_duration_s":0.001486,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000697,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000789,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00063774,"exclusive_lock_hold_duration_s":0.00212076,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.015838,"completed_at":"2026-10-04T11:30:05.213Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000505,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.474409] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.215Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"da28164740884ac6c7b2779d","created_at":"2026-10-04T11:30:02.877Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80d39d646a2a5b276185d0d12bfc0467","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.942Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-da28164740884ac6c7b2779d: start","job_status":"start","queue_duration_s":2.273098,"scheduling_latency_s":2.273116} vm-test-run-nixbot-gitlab> gitlab # [ 225.489663] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.239Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"da28164740884ac6c7b2779d","created_at":"2026-10-04T11:30:02.877Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80d39d646a2a5b276185d0d12bfc0467","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.942Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-da28164740884ac6c7b2779d: done: 0.024375 sec","job_status":"done","queue_duration_s":2.273098,"scheduling_latency_s":2.273116,"redis_calls":12,"redis_duration_s":0.002063,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00031,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000763,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00099,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00049358,"exclusive_lock_hold_duration_s":0.00526558,"duration_s":0.024375,"completed_at":"2026-10-04T11:30:05.239Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002722,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.533841] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.240Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"6542c475c9c87891e466affb","created_at":"2026-10-04T11:30:02.957Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21072d3f36ca18d5ebc22c92dc63f821","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.963Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-6542c475c9c87891e466affb: start","job_status":"start","queue_duration_s":2.277211,"scheduling_latency_s":2.277228} vm-test-run-nixbot-gitlab> gitlab # [ 225.548753] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.254Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"6542c475c9c87891e466affb","created_at":"2026-10-04T11:30:02.957Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21072d3f36ca18d5ebc22c92dc63f821","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:02.963Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-6542c475c9c87891e466affb: done: 0.013456 sec","job_status":"done","queue_duration_s":2.277211,"scheduling_latency_s":2.277228,"redis_calls":7,"redis_duration_s":0.0009699999999999999,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0007,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00027,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008885,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013456,"completed_at":"2026-10-04T11:30:05.254Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000459,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.591737] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.255Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"7595acb3cbce97952cb12bb2","created_at":"2026-10-04T11:30:02.994Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"00780d09ca10d2b1d9f5943a27e279a5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.003Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-7595acb3cbce97952cb12bb2: start","job_status":"start","queue_duration_s":2.252259,"scheduling_latency_s":2.252277} vm-test-run-nixbot-gitlab> gitlab # [ 225.607045] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.262Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 225.610522] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.268Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"7595acb3cbce97952cb12bb2","created_at":"2026-10-04T11:30:02.994Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"00780d09ca10d2b1d9f5943a27e279a5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.003Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-7595acb3cbce97952cb12bb2: done: 0.012729 sec","job_status":"done","queue_duration_s":2.252259,"scheduling_latency_s":2.252277,"redis_calls":7,"redis_duration_s":0.0009760000000000001,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000697,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000279,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012729,"completed_at":"2026-10-04T11:30:05.268Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.652803] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.269Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"70476287980e11ea5b0247c2","created_at":"2026-10-04T11:30:03.018Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20756b6379dc04dab552dab9f2b092d2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.023Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-70476287980e11ea5b0247c2: start","job_status":"start","queue_duration_s":2.246204,"scheduling_latency_s":2.24622} vm-test-run-nixbot-gitlab> gitlab # [ 225.666901] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.285Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"70476287980e11ea5b0247c2","created_at":"2026-10-04T11:30:03.018Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20756b6379dc04dab552dab9f2b092d2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.023Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-70476287980e11ea5b0247c2: done: 0.015448 sec","job_status":"done","queue_duration_s":2.246204,"scheduling_latency_s":2.24622,"redis_calls":9,"redis_duration_s":0.001617,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000709,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000908,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00047828,"exclusive_lock_hold_duration_s":0.00248478,"duration_s":0.015448,"completed_at":"2026-10-04T11:30:05.284Z","load_balancing_strategy":"primary","db_duration_s":0.000492,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.708467] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.286Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"aada47212820b58d7edbd7ae","created_at":"2026-10-04T11:30:03.052Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d3ca93410b358344d2fefe000a86a1a0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.060Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-aada47212820b58d7edbd7ae: start","job_status":"start","queue_duration_s":2.225166,"scheduling_latency_s":2.225182} vm-test-run-nixbot-gitlab> gitlab # [ 225.723304] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.293Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 225.727208] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.298Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"aada47212820b58d7edbd7ae","created_at":"2026-10-04T11:30:03.052Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d3ca93410b358344d2fefe000a86a1a0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.060Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-aada47212820b58d7edbd7ae: done: 0.012295 sec","job_status":"done","queue_duration_s":2.225166,"scheduling_latency_s":2.225182,"redis_calls":7,"redis_duration_s":0.000968,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000696,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012295,"completed_at":"2026-10-04T11:30:05.298Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.769506] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.299Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"ed42ef9c294110c367277d16","created_at":"2026-10-04T11:30:03.081Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dee6964b7c64478f0f595030ea6a8fe8","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.112Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-ed42ef9c294110c367277d16: start","job_status":"start","queue_duration_s":2.187484,"scheduling_latency_s":2.187502} vm-test-run-nixbot-gitlab> gitlab # [ 225.785708] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.325Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"ed42ef9c294110c367277d16","created_at":"2026-10-04T11:30:03.081Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dee6964b7c64478f0f595030ea6a8fe8","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.112Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-ed42ef9c294110c367277d16: done: 0.026059 sec","job_status":"done","queue_duration_s":2.187484,"scheduling_latency_s":2.187502,"redis_calls":13,"redis_duration_s":0.00196,"redis_read_bytes":10,"redis_write_bytes":1343,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000464,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000776,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.00072,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":368,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024264,"extra.packages_cleanup_delete_orphaned_dependencies_worker.last_processed_packages_dependency_id":0,"extra.packages_cleanup_delete_orphaned_dependencies_worker.deleted_rows_count":0,"duration_s":0.026059,"completed_at":"2026-10-04T11:30:05.325Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003233,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.831893] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.326Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"5d505dbade5b717b5a1f26bc","created_at":"2026-10-04T11:30:03.127Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f47b723c3794d6ee6c2217b11a68b7a6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-5d505dbade5b717b5a1f26bc: start","job_status":"start","queue_duration_s":2.192636,"scheduling_latency_s":2.192653} vm-test-run-nixbot-gitlab> gitlab # [ 225.847023] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.536Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"5d505dbade5b717b5a1f26bc","created_at":"2026-10-04T11:30:03.127Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f47b723c3794d6ee6c2217b11a68b7a6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.134Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-5d505dbade5b717b5a1f26bc: done: 0.209766 sec","job_status":"done","queue_duration_s":2.192636,"scheduling_latency_s":2.192653,"redis_calls":9,"redis_duration_s":0.001493,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000822,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000671,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":129,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.173335,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00045824,"exclusive_lock_hold_duration_s":0.1954695,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.209766,"completed_at":"2026-10-04T11:30:05.536Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.044568,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.891885] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.537Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"edfaf15ba6bdc4400fee6bee","created_at":"2026-10-04T11:30:03.137Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fd5c4b8d968eb45c91921c700f391087","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.142Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-edfaf15ba6bdc4400fee6bee: start","job_status":"start","queue_duration_s":2.394773,"scheduling_latency_s":2.394791} vm-test-run-nixbot-gitlab> gitlab # [ 225.907274] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.550Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"edfaf15ba6bdc4400fee6bee","created_at":"2026-10-04T11:30:03.137Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fd5c4b8d968eb45c91921c700f391087","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.142Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-edfaf15ba6bdc4400fee6bee: done: 0.012936 sec","job_status":"done","queue_duration_s":2.394773,"scheduling_latency_s":2.394791,"redis_calls":7,"redis_duration_s":0.000995,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000717,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000278,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008389,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012936,"completed_at":"2026-10-04T11:30:05.550Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.949755] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.551Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"b7d2d7343d23d2271b5722f8","created_at":"2026-10-04T11:30:03.147Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1cf4233e0468b4f116b2c5e9494dddb5","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.198Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-b7d2d7343d23d2271b5722f8: start","job_status":"start","queue_duration_s":2.353653,"scheduling_latency_s":2.353671} vm-test-run-nixbot-gitlab> gitlab # [ 225.963474] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.578Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"b7d2d7343d23d2271b5722f8","created_at":"2026-10-04T11:30:03.147Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1cf4233e0468b4f116b2c5e9494dddb5","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.198Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-b7d2d7343d23d2271b5722f8: done: 0.026708 sec","job_status":"done","queue_duration_s":2.353653,"scheduling_latency_s":2.353671,"redis_calls":10,"redis_duration_s":0.0013440000000000001,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000295,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000764,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000285,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024674,"duration_s":0.026708,"completed_at":"2026-10-04T11:30:05.578Z","load_balancing_strategy":"primary","db_duration_s":0.004263,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.007756] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.579Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"8b3f698c639a375b0a36ac80","created_at":"2026-10-04T11:30:03.213Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb49068d038b57d70642a370d7339ba9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.219Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-8b3f698c639a375b0a36ac80: start","job_status":"start","queue_duration_s":2.359877,"scheduling_latency_s":2.359895} vm-test-run-nixbot-gitlab> gitlab # [ 226.023147] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.616Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"8b3f698c639a375b0a36ac80","created_at":"2026-10-04T11:30:03.213Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb49068d038b57d70642a370d7339ba9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.219Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-8b3f698c639a375b0a36ac80: done: 0.036427 sec","job_status":"done","queue_duration_s":2.359877,"scheduling_latency_s":2.359895,"redis_calls":9,"redis_duration_s":0.001346,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000705,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000641,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00047766,"exclusive_lock_hold_duration_s":0.02292948,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.036427,"completed_at":"2026-10-04T11:30:05.616Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010667,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.068310] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.617Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"419067d9ed879443d322e9ba","created_at":"2026-10-04T11:30:03.262Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd71cfb5dac44daa49713ea6c2269458","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.268Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-419067d9ed879443d322e9ba: start","job_status":"start","queue_duration_s":2.348419,"scheduling_latency_s":2.348437} vm-test-run-nixbot-gitlab> gitlab # [ 226.083634] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.624Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 226.087342] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.629Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"419067d9ed879443d322e9ba","created_at":"2026-10-04T11:30:03.262Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd71cfb5dac44daa49713ea6c2269458","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.268Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-419067d9ed879443d322e9ba: done: 0.012302 sec","job_status":"done","queue_duration_s":2.348419,"scheduling_latency_s":2.348437,"redis_calls":7,"redis_duration_s":0.0009350000000000001,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000666,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000269,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012302,"completed_at":"2026-10-04T11:30:05.629Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.130071] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.631Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"f03c311ad67af3059c824af3","created_at":"2026-10-04T11:30:03.288Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91edef4924192243ac8b5bdaf566e378","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.306Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-f03c311ad67af3059c824af3: start","job_status":"start","queue_duration_s":2.324815,"scheduling_latency_s":2.324854} vm-test-run-nixbot-gitlab> gitlab # [ 226.144154] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.682Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"f03c311ad67af3059c824af3","created_at":"2026-10-04T11:30:03.288Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91edef4924192243ac8b5bdaf566e378","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.306Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-f03c311ad67af3059c824af3: done: 0.05176 sec","job_status":"done","queue_duration_s":2.324815,"scheduling_latency_s":2.324854,"redis_calls":10,"redis_duration_s":0.001472,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000326,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000798,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000348,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002458,"extra.pages_deactivate_expired_deployments_cron_worker.deactivate_expired_pages_deployments":{"deactivated_deployments":0,"duration":0.00322184},"duration_s":0.05176,"completed_at":"2026-10-04T11:30:05.682Z","load_balancing_strategy":"primary","db_duration_s":0.019368,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.189793] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.684Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"bf487861924f834ef6152853","created_at":"2026-10-04T11:30:03.316Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c32aa752cbb6b55f5c5b9f5b76d8dd88","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.330Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-bf487861924f834ef6152853: start","job_status":"start","queue_duration_s":2.353112,"scheduling_latency_s":2.35313} vm-test-run-nixbot-gitlab> gitlab # [ 226.203526] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.705Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"bf487861924f834ef6152853","created_at":"2026-10-04T11:30:03.316Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c32aa752cbb6b55f5c5b9f5b76d8dd88","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.330Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-bf487861924f834ef6152853: done: 0.021609 sec","job_status":"done","queue_duration_s":2.353112,"scheduling_latency_s":2.35313,"redis_calls":10,"redis_duration_s":0.001385,"redis_read_bytes":9,"redis_write_bytes":1087,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000276,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000795,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000314,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002552,"duration_s":0.021609,"completed_at":"2026-10-04T11:30:05.705Z","load_balancing_strategy":"primary","db_duration_s":0.000506,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.246284] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.706Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"10ae5c22296fd7ac7a826384","created_at":"2026-10-04T11:30:03.337Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4608f64a67cb48d3643cbb04414ae28f","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.346Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-10ae5c22296fd7ac7a826384: start","job_status":"start","queue_duration_s":2.360222,"scheduling_latency_s":2.360239} vm-test-run-nixbot-gitlab> gitlab # [ 226.260305] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.727Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"10ae5c22296fd7ac7a826384","created_at":"2026-10-04T11:30:03.337Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4608f64a67cb48d3643cbb04414ae28f","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:03.346Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-10ae5c22296fd7ac7a826384: done: 0.020224 sec","job_status":"done","queue_duration_s":2.360222,"scheduling_latency_s":2.360239,"redis_calls":10,"redis_duration_s":0.001437,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000305,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000807,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000325,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024156,"duration_s":0.020224,"completed_at":"2026-10-04T11:30:05.727Z","load_balancing_strategy":"primary","db_duration_s":0.000879,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.303893] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.728Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41660169c5c0e5cc349cf54b","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41660169c5c0e5cc349cf54b: start","job_status":"start","queue_duration_s":0.990405,"scheduling_latency_s":0.990422} vm-test-run-nixbot-gitlab> gitlab # [ 226.317269] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.742Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41660169c5c0e5cc349cf54b","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41660169c5c0e5cc349cf54b: done: 0.014762 sec","job_status":"done","queue_duration_s":0.990405,"scheduling_latency_s":0.990422,"redis_calls":13,"redis_duration_s":0.0015400000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014762,"completed_at":"2026-10-04T11:30:05.742Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.357634] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.744Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"736853d50e170f75c42f187c","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-736853d50e170f75c42f187c: start","job_status":"start","queue_duration_s":1.006325,"scheduling_latency_s":1.006342} vm-test-run-nixbot-gitlab> gitlab # [ 226.370450] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"736853d50e170f75c42f187c","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-736853d50e170f75c42f187c: done: 0.014408 sec","job_status":"done","queue_duration_s":1.006325,"scheduling_latency_s":1.006342,"redis_calls":13,"redis_duration_s":0.001486,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014408,"completed_at":"2026-10-04T11:30:05.758Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.412505] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2797a5e56cbf1f5ac48a9f33","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2797a5e56cbf1f5ac48a9f33: start","job_status":"start","queue_duration_s":1.022072,"scheduling_latency_s":1.022089} vm-test-run-nixbot-gitlab> gitlab # [ 226.425407] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2797a5e56cbf1f5ac48a9f33","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2797a5e56cbf1f5ac48a9f33: done: 0.014185 sec","job_status":"done","queue_duration_s":1.022072,"scheduling_latency_s":1.022089,"redis_calls":13,"redis_duration_s":0.001488,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014185,"completed_at":"2026-10-04T11:30:05.773Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.465898] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd030df8a6dfab26c04107f9","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd030df8a6dfab26c04107f9: start","job_status":"start","queue_duration_s":1.037381,"scheduling_latency_s":1.037398} vm-test-run-nixbot-gitlab> gitlab # [ 226.478819] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd030df8a6dfab26c04107f9","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd030df8a6dfab26c04107f9: done: 0.014398 sec","job_status":"done","queue_duration_s":1.037381,"scheduling_latency_s":1.037398,"redis_calls":13,"redis_duration_s":0.001477,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014398,"completed_at":"2026-10-04T11:30:05.789Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.519472] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4552307a8a52592a99b847d4","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4552307a8a52592a99b847d4: start","job_status":"start","queue_duration_s":1.05287,"scheduling_latency_s":1.052886} vm-test-run-nixbot-gitlab> gitlab # [ 226.532326] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4552307a8a52592a99b847d4","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4552307a8a52592a99b847d4: done: 0.014382 sec","job_status":"done","queue_duration_s":1.05287,"scheduling_latency_s":1.052886,"redis_calls":13,"redis_duration_s":0.001508,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009897,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014382,"completed_at":"2026-10-04T11:30:05.805Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.572838] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1aca024bd4c6c938e097d965","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1aca024bd4c6c938e097d965: start","job_status":"start","queue_duration_s":1.068365,"scheduling_latency_s":1.068382} vm-test-run-nixbot-gitlab> gitlab # [ 226.585600] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.821Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1aca024bd4c6c938e097d965","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1aca024bd4c6c938e097d965: done: 0.014972 sec","job_status":"done","queue_duration_s":1.068365,"scheduling_latency_s":1.068382,"redis_calls":13,"redis_duration_s":0.001488,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014972,"completed_at":"2026-10-04T11:30:05.821Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.626804] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.822Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04b40e9573bd00b6ac1f3418","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04b40e9573bd00b6ac1f3418: start","job_status":"start","queue_duration_s":1.08444,"scheduling_latency_s":1.084455} vm-test-run-nixbot-gitlab> gitlab # [ 226.639703] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.836Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04b40e9573bd00b6ac1f3418","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04b40e9573bd00b6ac1f3418: done: 0.014238 sec","job_status":"done","queue_duration_s":1.08444,"scheduling_latency_s":1.084455,"redis_calls":13,"redis_duration_s":0.0014810000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014238,"completed_at":"2026-10-04T11:30:05.836Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.680628] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e90e5aceb1324a581b37a78","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e90e5aceb1324a581b37a78: start","job_status":"start","queue_duration_s":1.099786,"scheduling_latency_s":1.099803} vm-test-run-nixbot-gitlab> gitlab # [ 226.693605] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.852Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e90e5aceb1324a581b37a78","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e90e5aceb1324a581b37a78: done: 0.014567 sec","job_status":"done","queue_duration_s":1.099786,"scheduling_latency_s":1.099803,"redis_calls":13,"redis_duration_s":0.001567,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014567,"completed_at":"2026-10-04T11:30:05.852Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.734566] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.853Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33e389806e8b0242b54a3f47","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33e389806e8b0242b54a3f47: start","job_status":"start","queue_duration_s":1.115479,"scheduling_latency_s":1.115496} vm-test-run-nixbot-gitlab> gitlab # [ 226.747556] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33e389806e8b0242b54a3f47","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33e389806e8b0242b54a3f47: done: 0.014377 sec","job_status":"done","queue_duration_s":1.115479,"scheduling_latency_s":1.115496,"redis_calls":13,"redis_duration_s":0.001501,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014377,"completed_at":"2026-10-04T11:30:05.867Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.788358] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b79f559aa9e242bbf3e49c4","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b79f559aa9e242bbf3e49c4: start","job_status":"start","queue_duration_s":1.130987,"scheduling_latency_s":1.131005} vm-test-run-nixbot-gitlab> gitlab # [ 226.802267] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.883Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b79f559aa9e242bbf3e49c4","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b79f559aa9e242bbf3e49c4: done: 0.014729 sec","job_status":"done","queue_duration_s":1.130987,"scheduling_latency_s":1.131005,"redis_calls":13,"redis_duration_s":0.00147,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014729,"completed_at":"2026-10-04T11:30:05.883Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.843026] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29ede7facea46de2b578b70e","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29ede7facea46de2b578b70e: start","job_status":"start","queue_duration_s":1.146845,"scheduling_latency_s":1.146862} vm-test-run-nixbot-gitlab> gitlab # [ 226.856273] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.899Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29ede7facea46de2b578b70e","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29ede7facea46de2b578b70e: done: 0.014436 sec","job_status":"done","queue_duration_s":1.146845,"scheduling_latency_s":1.146862,"redis_calls":13,"redis_duration_s":0.001591,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001044,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014436,"completed_at":"2026-10-04T11:30:05.899Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.897274] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4b0911a845686c434ce50c0","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4b0911a845686c434ce50c0: start","job_status":"start","queue_duration_s":1.162433,"scheduling_latency_s":1.16245} vm-test-run-nixbot-gitlab> gitlab # [ 226.910240] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.914Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4b0911a845686c434ce50c0","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4b0911a845686c434ce50c0: done: 0.014543 sec","job_status":"done","queue_duration_s":1.162433,"scheduling_latency_s":1.16245,"redis_calls":13,"redis_duration_s":0.001711,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000756,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014543,"completed_at":"2026-10-04T11:30:05.914Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.951141] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47c9a61961551d1ffa315ff7","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47c9a61961551d1ffa315ff7: start","job_status":"start","queue_duration_s":1.178113,"scheduling_latency_s":1.178131} vm-test-run-nixbot-gitlab> gitlab # [ 226.964144] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.930Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47c9a61961551d1ffa315ff7","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47c9a61961551d1ffa315ff7: done: 0.014954 sec","job_status":"done","queue_duration_s":1.178113,"scheduling_latency_s":1.178131,"redis_calls":13,"redis_duration_s":0.001881,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001342,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014954,"completed_at":"2026-10-04T11:30:05.930Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.005825] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3053b574cff95e5100d5448","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3053b574cff95e5100d5448: start","job_status":"start","queue_duration_s":1.194284,"scheduling_latency_s":1.194625} vm-test-run-nixbot-gitlab> gitlab # [ 227.018629] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3053b574cff95e5100d5448","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3053b574cff95e5100d5448: done: 0.014871 sec","job_status":"done","queue_duration_s":1.194284,"scheduling_latency_s":1.194625,"redis_calls":13,"redis_duration_s":0.00151,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014871,"completed_at":"2026-10-04T11:30:05.946Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.059248] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.948Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8176071caa3c70467fec2c31","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8176071caa3c70467fec2c31: start","job_status":"start","queue_duration_s":1.210355,"scheduling_latency_s":1.210372} vm-test-run-nixbot-gitlab> gitlab # [ 227.072301] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.963Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8176071caa3c70467fec2c31","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8176071caa3c70467fec2c31: done: 0.014934 sec","job_status":"done","queue_duration_s":1.210355,"scheduling_latency_s":1.210372,"redis_calls":13,"redis_duration_s":0.001621,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001068,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014934,"completed_at":"2026-10-04T11:30:05.963Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.112850] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8466ff6f0da8c3f70cfe5da4","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8466ff6f0da8c3f70cfe5da4: start","job_status":"start","queue_duration_s":1.226436,"scheduling_latency_s":1.226464} vm-test-run-nixbot-gitlab> gitlab # [ 227.125723] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8466ff6f0da8c3f70cfe5da4","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8466ff6f0da8c3f70cfe5da4: done: 0.014198 sec","job_status":"done","queue_duration_s":1.226436,"scheduling_latency_s":1.226464,"redis_calls":13,"redis_duration_s":0.001509,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009586,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014198,"completed_at":"2026-10-04T11:30:05.978Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.166316] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ec3e7a0480cc8148e46fe4d","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ec3e7a0480cc8148e46fe4d: start","job_status":"start","queue_duration_s":1.241958,"scheduling_latency_s":1.241976} vm-test-run-nixbot-gitlab> gitlab # [ 227.179099] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ec3e7a0480cc8148e46fe4d","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ec3e7a0480cc8148e46fe4d: done: 0.014734 sec","job_status":"done","queue_duration_s":1.241958,"scheduling_latency_s":1.241976,"redis_calls":13,"redis_duration_s":0.0014750000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009628,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014734,"completed_at":"2026-10-04T11:30:05.994Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.220871] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:05.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63519f3db2c7536b699d445c","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63519f3db2c7536b699d445c: start","job_status":"start","queue_duration_s":1.257656,"scheduling_latency_s":1.257674} vm-test-run-nixbot-gitlab> gitlab # [ 227.233614] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.009Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63519f3db2c7536b699d445c","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63519f3db2c7536b699d445c: done: 0.014372 sec","job_status":"done","queue_duration_s":1.257656,"scheduling_latency_s":1.257674,"redis_calls":13,"redis_duration_s":0.001497,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014372,"completed_at":"2026-10-04T11:30:06.009Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.273962] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57c14b10366032e9b4dcfb27","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57c14b10366032e9b4dcfb27: start","job_status":"start","queue_duration_s":1.273254,"scheduling_latency_s":1.273272} vm-test-run-nixbot-gitlab> gitlab # [ 227.286690] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.027Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57c14b10366032e9b4dcfb27","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57c14b10366032e9b4dcfb27: done: 0.016476 sec","job_status":"done","queue_duration_s":1.273254,"scheduling_latency_s":1.273272,"redis_calls":13,"redis_duration_s":0.0018930000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000607,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001286,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010967,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016476,"completed_at":"2026-10-04T11:30:06.027Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.327677] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c305bd7700fabeb2ad0f9854","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c305bd7700fabeb2ad0f9854: start","job_status":"start","queue_duration_s":1.29105,"scheduling_latency_s":1.291068} vm-test-run-nixbot-gitlab> gitlab # [ 227.340831] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.043Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c305bd7700fabeb2ad0f9854","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c305bd7700fabeb2ad0f9854: done: 0.015124 sec","job_status":"done","queue_duration_s":1.29105,"scheduling_latency_s":1.291068,"redis_calls":13,"redis_duration_s":0.0015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009899,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015124,"completed_at":"2026-10-04T11:30:06.043Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.381325] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.045Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"672218db40a18f93abe16f1a","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-672218db40a18f93abe16f1a: start","job_status":"start","queue_duration_s":1.307889,"scheduling_latency_s":1.307912} vm-test-run-nixbot-gitlab> gitlab # [ 227.394172] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"672218db40a18f93abe16f1a","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-672218db40a18f93abe16f1a: done: 0.015027 sec","job_status":"done","queue_duration_s":1.307889,"scheduling_latency_s":1.307912,"redis_calls":13,"redis_duration_s":0.00151,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015027,"completed_at":"2026-10-04T11:30:06.060Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.435734] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6b8f155c10a9fa69e2fdc7f","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6b8f155c10a9fa69e2fdc7f: start","job_status":"start","queue_duration_s":1.324038,"scheduling_latency_s":1.324056} vm-test-run-nixbot-gitlab> gitlab # [ 227.448695] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.076Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6b8f155c10a9fa69e2fdc7f","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6b8f155c10a9fa69e2fdc7f: done: 0.014474 sec","job_status":"done","queue_duration_s":1.324038,"scheduling_latency_s":1.324056,"redis_calls":13,"redis_duration_s":0.0015040000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014474,"completed_at":"2026-10-04T11:30:06.076Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.489484] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.077Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73bd8abc00705bb4c9b14d0f","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73bd8abc00705bb4c9b14d0f: start","job_status":"start","queue_duration_s":1.339685,"scheduling_latency_s":1.339703} vm-test-run-nixbot-gitlab> gitlab # [ 227.502290] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.092Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73bd8abc00705bb4c9b14d0f","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73bd8abc00705bb4c9b14d0f: done: 0.015118 sec","job_status":"done","queue_duration_s":1.339685,"scheduling_latency_s":1.339703,"redis_calls":13,"redis_duration_s":0.00156,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010591,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015118,"completed_at":"2026-10-04T11:30:06.092Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.542836] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.093Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91178049117d490dfd25f8aa","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91178049117d490dfd25f8aa: start","job_status":"start","queue_duration_s":1.355969,"scheduling_latency_s":1.355987} vm-test-run-nixbot-gitlab> gitlab # [ 227.556248] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91178049117d490dfd25f8aa","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91178049117d490dfd25f8aa: done: 0.015561 sec","job_status":"done","queue_duration_s":1.355969,"scheduling_latency_s":1.355987,"redis_calls":13,"redis_duration_s":0.00153,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015561,"completed_at":"2026-10-04T11:30:06.109Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.597154] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.110Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"267a17ef958186b3b7172987","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-267a17ef958186b3b7172987: start","job_status":"start","queue_duration_s":1.372673,"scheduling_latency_s":1.37269} vm-test-run-nixbot-gitlab> gitlab # [ 227.610945] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"267a17ef958186b3b7172987","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-267a17ef958186b3b7172987: done: 0.016951 sec","job_status":"done","queue_duration_s":1.372673,"scheduling_latency_s":1.37269,"redis_calls":13,"redis_duration_s":0.0016120000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000594,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016951,"completed_at":"2026-10-04T11:30:06.127Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.652576] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.128Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"343117114c4db9f5621366dd","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-343117114c4db9f5621366dd: start","job_status":"start","queue_duration_s":1.390819,"scheduling_latency_s":1.390836} vm-test-run-nixbot-gitlab> gitlab # [ 227.665402] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.145Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"343117114c4db9f5621366dd","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-343117114c4db9f5621366dd: done: 0.01707 sec","job_status":"done","queue_duration_s":1.390819,"scheduling_latency_s":1.390836,"redis_calls":13,"redis_duration_s":0.002221,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000573,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001648,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011296,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01707,"completed_at":"2026-10-04T11:30:06.145Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.705730] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21df6f2838ce82303e52d55c","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21df6f2838ce82303e52d55c: start","job_status":"start","queue_duration_s":1.409077,"scheduling_latency_s":1.409094} vm-test-run-nixbot-gitlab> gitlab # [ 227.718798] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.163Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21df6f2838ce82303e52d55c","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21df6f2838ce82303e52d55c: done: 0.016946 sec","job_status":"done","queue_duration_s":1.409077,"scheduling_latency_s":1.409094,"redis_calls":13,"redis_duration_s":0.001591,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000607,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016946,"completed_at":"2026-10-04T11:30:06.163Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.759485] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfb923461269b47ea66864c4","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfb923461269b47ea66864c4: start","job_status":"start","queue_duration_s":1.427172,"scheduling_latency_s":1.427189} vm-test-run-nixbot-gitlab> gitlab # [ 227.772421] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfb923461269b47ea66864c4","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfb923461269b47ea66864c4: done: 0.016397 sec","job_status":"done","queue_duration_s":1.427172,"scheduling_latency_s":1.427189,"redis_calls":13,"redis_duration_s":0.001559,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000594,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011217,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016397,"completed_at":"2026-10-04T11:30:06.181Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.813986] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"845a0b95d4f2845ade7ee5df","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-845a0b95d4f2845ade7ee5df: start","job_status":"start","queue_duration_s":1.445305,"scheduling_latency_s":1.44536} vm-test-run-nixbot-gitlab> gitlab # [ 227.826823] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"845a0b95d4f2845ade7ee5df","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-845a0b95d4f2845ade7ee5df: done: 0.016746 sec","job_status":"done","queue_duration_s":1.445305,"scheduling_latency_s":1.44536,"redis_calls":13,"redis_duration_s":0.00167,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016746,"completed_at":"2026-10-04T11:30:06.199Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.867870] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9679138b19ecafe249b2f3bf","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9679138b19ecafe249b2f3bf: start","job_status":"start","queue_duration_s":1.463773,"scheduling_latency_s":1.463791} vm-test-run-nixbot-gitlab> gitlab # [ 227.881124] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9679138b19ecafe249b2f3bf","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9679138b19ecafe249b2f3bf: done: 0.016641 sec","job_status":"done","queue_duration_s":1.463773,"scheduling_latency_s":1.463791,"redis_calls":13,"redis_duration_s":0.0015830000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000573,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00101,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016641,"completed_at":"2026-10-04T11:30:06.218Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.921867] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d71150b0139e8b668b1afc3","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d71150b0139e8b668b1afc3: start","job_status":"start","queue_duration_s":1.482138,"scheduling_latency_s":1.482157} vm-test-run-nixbot-gitlab> gitlab # [ 227.934639] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d71150b0139e8b668b1afc3","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d71150b0139e8b668b1afc3: done: 0.016653 sec","job_status":"done","queue_duration_s":1.482138,"scheduling_latency_s":1.482157,"redis_calls":13,"redis_duration_s":0.001568,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016653,"completed_at":"2026-10-04T11:30:06.236Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.975728] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.238Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6cbbfe5811272531b6ad149","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6cbbfe5811272531b6ad149: start","job_status":"start","queue_duration_s":1.500569,"scheduling_latency_s":1.500587} vm-test-run-nixbot-gitlab> gitlab # [ 227.989000] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6cbbfe5811272531b6ad149","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6cbbfe5811272531b6ad149: done: 0.016681 sec","job_status":"done","queue_duration_s":1.500569,"scheduling_latency_s":1.500587,"redis_calls":13,"redis_duration_s":0.0015899999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016681,"completed_at":"2026-10-04T11:30:06.254Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.030909] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"361d2e32d16dc27c0aa3e572","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-361d2e32d16dc27c0aa3e572: start","job_status":"start","queue_duration_s":1.519091,"scheduling_latency_s":1.51911} vm-test-run-nixbot-gitlab> gitlab # [ 228.043820] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"361d2e32d16dc27c0aa3e572","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-361d2e32d16dc27c0aa3e572: done: 0.01653 sec","job_status":"done","queue_duration_s":1.519091,"scheduling_latency_s":1.51911,"redis_calls":13,"redis_duration_s":0.002103,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001112,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01653,"completed_at":"2026-10-04T11:30:06.273Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.084726] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.274Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5071c2241ad36f4d350a11c","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5071c2241ad36f4d350a11c: start","job_status":"start","queue_duration_s":1.536742,"scheduling_latency_s":1.536759} vm-test-run-nixbot-gitlab> gitlab # [ 228.097854] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.290Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5071c2241ad36f4d350a11c","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5071c2241ad36f4d350a11c: done: 0.016368 sec","job_status":"done","queue_duration_s":1.536742,"scheduling_latency_s":1.536759,"redis_calls":13,"redis_duration_s":0.001624,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000568,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001056,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011147,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016368,"completed_at":"2026-10-04T11:30:06.290Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.138798] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.292Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3faa303a1b6c2c1eb07b5c32","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3faa303a1b6c2c1eb07b5c32: start","job_status":"start","queue_duration_s":1.554287,"scheduling_latency_s":1.554305} vm-test-run-nixbot-gitlab> gitlab # [ 228.151749] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.306Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3faa303a1b6c2c1eb07b5c32","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3faa303a1b6c2c1eb07b5c32: done: 0.014546 sec","job_status":"done","queue_duration_s":1.554287,"scheduling_latency_s":1.554305,"redis_calls":13,"redis_duration_s":0.001503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014546,"completed_at":"2026-10-04T11:30:06.306Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.192565] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.307Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63e377f243eee2552ff02af5","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63e377f243eee2552ff02af5: start","job_status":"start","queue_duration_s":1.57001,"scheduling_latency_s":1.570027} vm-test-run-nixbot-gitlab> gitlab # [ 228.205465] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.322Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63e377f243eee2552ff02af5","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63e377f243eee2552ff02af5: done: 0.014898 sec","job_status":"done","queue_duration_s":1.57001,"scheduling_latency_s":1.570027,"redis_calls":13,"redis_duration_s":0.001559,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014898,"completed_at":"2026-10-04T11:30:06.322Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.247105] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.323Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6455caa015a58046354d704","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6455caa015a58046354d704: start","job_status":"start","queue_duration_s":1.586093,"scheduling_latency_s":1.586112} vm-test-run-nixbot-gitlab> gitlab # [ 228.260307] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6455caa015a58046354d704","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6455caa015a58046354d704: done: 0.014609 sec","job_status":"done","queue_duration_s":1.586093,"scheduling_latency_s":1.586112,"redis_calls":13,"redis_duration_s":0.001553,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014609,"completed_at":"2026-10-04T11:30:06.338Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.301048] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.339Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f52ac04c367bb0f535ccd080","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f52ac04c367bb0f535ccd080: start","job_status":"start","queue_duration_s":1.601966,"scheduling_latency_s":1.601984} vm-test-run-nixbot-gitlab> gitlab # [ 228.313777] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.354Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f52ac04c367bb0f535ccd080","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f52ac04c367bb0f535ccd080: done: 0.014358 sec","job_status":"done","queue_duration_s":1.601966,"scheduling_latency_s":1.601984,"redis_calls":13,"redis_duration_s":0.0015110000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014358,"completed_at":"2026-10-04T11:30:06.354Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.354769] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6e81cc43263720a88e639a2","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6e81cc43263720a88e639a2: start","job_status":"start","queue_duration_s":1.61754,"scheduling_latency_s":1.617558} vm-test-run-nixbot-gitlab> gitlab # [ 228.367805] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6e81cc43263720a88e639a2","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6e81cc43263720a88e639a2: done: 0.01446 sec","job_status":"done","queue_duration_s":1.61754,"scheduling_latency_s":1.617558,"redis_calls":13,"redis_duration_s":0.001498,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000962,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01446,"completed_at":"2026-10-04T11:30:06.369Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.408700] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.371Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca8a33a6fb03f44b6a6f79ec","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca8a33a6fb03f44b6a6f79ec: start","job_status":"start","queue_duration_s":1.633168,"scheduling_latency_s":1.633187} vm-test-run-nixbot-gitlab> gitlab # [ 228.421536] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca8a33a6fb03f44b6a6f79ec","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca8a33a6fb03f44b6a6f79ec: done: 0.014615 sec","job_status":"done","queue_duration_s":1.633168,"scheduling_latency_s":1.633187,"redis_calls":13,"redis_duration_s":0.0016589999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000573,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001086,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014615,"completed_at":"2026-10-04T11:30:06.385Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.463380] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73a6603fc77b683a6fe1a530","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73a6603fc77b683a6fe1a530: start","job_status":"start","queue_duration_s":1.648953,"scheduling_latency_s":1.64897} vm-test-run-nixbot-gitlab> gitlab # [ 228.476719] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.401Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73a6603fc77b683a6fe1a530","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73a6603fc77b683a6fe1a530: done: 0.014287 sec","job_status":"done","queue_duration_s":1.648953,"scheduling_latency_s":1.64897,"redis_calls":13,"redis_duration_s":0.001492,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014287,"completed_at":"2026-10-04T11:30:06.400Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.517283] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.402Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1cf24f541ac8e9367ccdc7a1","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1cf24f541ac8e9367ccdc7a1: start","job_status":"start","queue_duration_s":1.664399,"scheduling_latency_s":1.664418} vm-test-run-nixbot-gitlab> gitlab # [ 228.530308] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1cf24f541ac8e9367ccdc7a1","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1cf24f541ac8e9367ccdc7a1: done: 0.014832 sec","job_status":"done","queue_duration_s":1.664399,"scheduling_latency_s":1.664418,"redis_calls":13,"redis_duration_s":0.001508,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000967,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014832,"completed_at":"2026-10-04T11:30:06.416Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.571526] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93d6706b9668bef0c1ed4f76","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93d6706b9668bef0c1ed4f76: start","job_status":"start","queue_duration_s":1.680392,"scheduling_latency_s":1.680409} vm-test-run-nixbot-gitlab> gitlab # [ 228.584691] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93d6706b9668bef0c1ed4f76","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93d6706b9668bef0c1ed4f76: done: 0.01436 sec","job_status":"done","queue_duration_s":1.680392,"scheduling_latency_s":1.680409,"redis_calls":13,"redis_duration_s":0.001546,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000995,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009808,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01436,"completed_at":"2026-10-04T11:30:06.432Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.625436] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cdf767d28af2cf20ca444df4","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cdf767d28af2cf20ca444df4: start","job_status":"start","queue_duration_s":1.696023,"scheduling_latency_s":1.696042} vm-test-run-nixbot-gitlab> gitlab # [ 228.639322] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.448Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cdf767d28af2cf20ca444df4","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cdf767d28af2cf20ca444df4: done: 0.014315 sec","job_status":"done","queue_duration_s":1.696023,"scheduling_latency_s":1.696042,"redis_calls":13,"redis_duration_s":0.0015249999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000561,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014315,"completed_at":"2026-10-04T11:30:06.448Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.680572] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c0bba4dd3cc207bb3beaea8","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c0bba4dd3cc207bb3beaea8: start","job_status":"start","queue_duration_s":1.711534,"scheduling_latency_s":1.711553} vm-test-run-nixbot-gitlab> gitlab # [ 228.693278] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.463Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c0bba4dd3cc207bb3beaea8","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c0bba4dd3cc207bb3beaea8: done: 0.014381 sec","job_status":"done","queue_duration_s":1.711534,"scheduling_latency_s":1.711553,"redis_calls":13,"redis_duration_s":0.001482,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014381,"completed_at":"2026-10-04T11:30:06.463Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.733992] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b34f979216f229a5b162e31","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b34f979216f229a5b162e31: start","job_status":"start","queue_duration_s":1.727111,"scheduling_latency_s":1.727129} vm-test-run-nixbot-gitlab> gitlab # [ 228.746811] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.479Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b34f979216f229a5b162e31","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b34f979216f229a5b162e31: done: 0.014186 sec","job_status":"done","queue_duration_s":1.727111,"scheduling_latency_s":1.727129,"redis_calls":13,"redis_duration_s":0.001484,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009732,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014186,"completed_at":"2026-10-04T11:30:06.479Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.787382] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4e4d28892ddd00e1c3c5c5e","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4e4d28892ddd00e1c3c5c5e: start","job_status":"start","queue_duration_s":1.742564,"scheduling_latency_s":1.742582} vm-test-run-nixbot-gitlab> gitlab # [ 228.800426] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.494Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4e4d28892ddd00e1c3c5c5e","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4e4d28892ddd00e1c3c5c5e: done: 0.014416 sec","job_status":"done","queue_duration_s":1.742564,"scheduling_latency_s":1.742582,"redis_calls":13,"redis_duration_s":0.001569,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000591,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014416,"completed_at":"2026-10-04T11:30:06.494Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.842434] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.496Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ad4eb53bee5f6e5846610b5","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ad4eb53bee5f6e5846610b5: start","job_status":"start","queue_duration_s":1.758203,"scheduling_latency_s":1.758221} vm-test-run-nixbot-gitlab> gitlab # [ 228.855484] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ad4eb53bee5f6e5846610b5","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ad4eb53bee5f6e5846610b5: done: 0.014297 sec","job_status":"done","queue_duration_s":1.758203,"scheduling_latency_s":1.758221,"redis_calls":13,"redis_duration_s":0.001577,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001031,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014297,"completed_at":"2026-10-04T11:30:06.510Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.896233] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.511Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9066a118077f49c9caa1ab44","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9066a118077f49c9caa1ab44: start","job_status":"start","queue_duration_s":1.773734,"scheduling_latency_s":1.773751} vm-test-run-nixbot-gitlab> gitlab # [ 228.909151] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.526Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9066a118077f49c9caa1ab44","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9066a118077f49c9caa1ab44: done: 0.014581 sec","job_status":"done","queue_duration_s":1.773734,"scheduling_latency_s":1.773751,"redis_calls":13,"redis_duration_s":0.0017209999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014581,"completed_at":"2026-10-04T11:30:06.526Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.950562] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.527Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d2554e7c5e0e861109e142f","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d2554e7c5e0e861109e142f: start","job_status":"start","queue_duration_s":1.789498,"scheduling_latency_s":1.789517} vm-test-run-nixbot-gitlab> gitlab # [ 228.963625] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.541Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d2554e7c5e0e861109e142f","created_at":"2026-10-04T11:30:04.657Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a4ff5fb6615f4f98","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"021fe4c93d5e496ed2fb3e8b94ab8bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.737Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d2554e7c5e0e861109e142f: done: 0.014452 sec","job_status":"done","queue_duration_s":1.789498,"scheduling_latency_s":1.789517,"redis_calls":13,"redis_duration_s":0.001518,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000555,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014452,"completed_at":"2026-10-04T11:30:06.541Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 229.004509] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.543Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"a5cf4e19e5f331559af0f8fb","created_at":"2026-10-04T11:30:04.946Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-6e6d671641cd4a6a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"53f9bffdd6527bfe41111fc40e5c8253","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.955Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-a5cf4e19e5f331559af0f8fb: start","job_status":"start","queue_duration_s":1.587601,"scheduling_latency_s":1.587619} vm-test-run-nixbot-gitlab> gitlab # [ 229.019915] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.567Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"a5cf4e19e5f331559af0f8fb","created_at":"2026-10-04T11:30:04.946Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-6e6d671641cd4a6a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"53f9bffdd6527bfe41111fc40e5c8253","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.955Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-a5cf4e19e5f331559af0f8fb: done: 0.024781 sec","job_status":"done","queue_duration_s":1.587601,"scheduling_latency_s":1.587619,"redis_calls":10,"redis_duration_s":0.001387,"redis_read_bytes":9,"redis_write_bytes":1117,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000291,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000816,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00028,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":222,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014949,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026882,"duration_s":0.024781,"completed_at":"2026-10-04T11:30:06.567Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003632,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 229.065588] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.569Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"6b9bfafb0e10f24a6b3ad22e","created_at":"2026-10-04T11:30:04.956Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-6e6d671641cd4a6a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"53f9bffdd6527bfe41111fc40e5c8253","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.960Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-6b9bfafb0e10f24a6b3ad22e: start","job_status":"start","queue_duration_s":1.608189,"scheduling_latency_s":1.608206} vm-test-run-nixbot-gitlab> gitlab # [ 229.081562] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.589Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"6b9bfafb0e10f24a6b3ad22e","created_at":"2026-10-04T11:30:04.956Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-6e6d671641cd4a6a","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"53f9bffdd6527bfe41111fc40e5c8253","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-10-04T11:30:04.960Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-6b9bfafb0e10f24a6b3ad22e: done: 0.020612 sec","job_status":"done","queue_duration_s":1.608189,"scheduling_latency_s":1.608206,"redis_calls":10,"redis_duration_s":0.001457,"redis_read_bytes":9,"redis_write_bytes":1133,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000301,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000858,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000298,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025264,"duration_s":0.020612,"completed_at":"2026-10-04T11:30:06.589Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001009,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 229.126110] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.590Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"60edf26dc691437fe4ded3d4","created_at":"2026-10-04T11:30:05.058Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-29d77d5f18744fbd","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"b20d2b0eadfc3af417057246635680a9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:05.074Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-60edf26dc691437fe4ded3d4: start","job_status":"start","queue_duration_s":1.515963,"scheduling_latency_s":1.51598} vm-test-run-nixbot-gitlab> gitlab # [ 229.139416] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.629Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"60edf26dc691437fe4ded3d4","created_at":"2026-10-04T11:30:05.058Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-29d77d5f18744fbd","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"b20d2b0eadfc3af417057246635680a9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:05.074Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-60edf26dc691437fe4ded3d4: done: 0.038764 sec","job_status":"done","queue_duration_s":1.515963,"scheduling_latency_s":1.51598,"redis_calls":18,"redis_duration_s":0.002981,"redis_read_bytes":39,"redis_write_bytes":2116,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00029,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000645,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":471,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.002046,"redis_shared_state_read_bytes":34,"redis_shared_state_write_bytes":1324,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022825,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026158,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":0,"duration_s":0.038764,"completed_at":"2026-10-04T11:30:06.629Z","load_balancing_strategy":"primary","db_duration_s":0.009304,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 229.184958] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"84bc962ff97f42be179bed20","created_at":"2026-10-04T11:30:05.058Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-29d77d5f18744fbd","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"b20d2b0eadfc3af417057246635680a9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:05.074Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-84bc962ff97f42be179bed20: start","job_status":"start","queue_duration_s":1.55605,"scheduling_latency_s":1.556069} vm-test-run-nixbot-gitlab> gitlab # [ 229.197902] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.656Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"84bc962ff97f42be179bed20","created_at":"2026-10-04T11:30:05.058Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-29d77d5f18744fbd","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"b20d2b0eadfc3af417057246635680a9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:05.074Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-84bc962ff97f42be179bed20: done: 0.025738 sec","job_status":"done","queue_duration_s":1.55605,"scheduling_latency_s":1.556069,"redis_calls":15,"redis_duration_s":0.001866,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000589,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.001277,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":4,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":4,"duration_s":0.025738,"completed_at":"2026-10-04T11:30:06.656Z","load_balancing_strategy":"primary","db_duration_s":0.002665,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 229.242419] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.657Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"4c4838f2396b64842026b4fe","created_at":"2026-10-04T11:30:05.058Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-29d77d5f18744fbd","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"b20d2b0eadfc3af417057246635680a9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:05.074Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-4c4838f2396b64842026b4fe: start","job_status":"start","queue_duration_s":1.582959,"scheduling_latency_s":1.582979} vm-test-run-nixbot-gitlab> gitlab # [ 229.255619] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"4c4838f2396b64842026b4fe","created_at":"2026-10-04T11:30:05.058Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-29d77d5f18744fbd","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"b20d2b0eadfc3af417057246635680a9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:05.074Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-4c4838f2396b64842026b4fe: done: 0.021924 sec","job_status":"done","queue_duration_s":1.582959,"scheduling_latency_s":1.582979,"redis_calls":15,"redis_duration_s":0.001758,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.001184,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":1,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":1,"duration_s":0.021924,"completed_at":"2026-10-04T11:30:06.679Z","load_balancing_strategy":"primary","db_duration_s":0.002517,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 229.299645] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.681Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"622edd69df77722627eb4bc5","created_at":"2026-10-04T11:30:05.058Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-29d77d5f18744fbd","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"b20d2b0eadfc3af417057246635680a9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:05.074Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-622edd69df77722627eb4bc5: start","job_status":"start","queue_duration_s":1.606256,"scheduling_latency_s":1.606276} vm-test-run-nixbot-gitlab> gitlab # [ 229.312880] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"622edd69df77722627eb4bc5","created_at":"2026-10-04T11:30:05.058Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-29d77d5f18744fbd","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"b20d2b0eadfc3af417057246635680a9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:05.074Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-622edd69df77722627eb4bc5: done: 0.027281 sec","job_status":"done","queue_duration_s":1.606256,"scheduling_latency_s":1.606276,"redis_calls":15,"redis_duration_s":0.004964,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000665,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.004299,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016476,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":0,"duration_s":0.027281,"completed_at":"2026-10-04T11:30:06.708Z","load_balancing_strategy":"primary","db_duration_s":0.003616,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 229.357049] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.709Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"a6520067a4c283c00c7519e3","created_at":"2026-10-04T11:30:05.058Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-29d77d5f18744fbd","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"b20d2b0eadfc3af417057246635680a9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:05.074Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-a6520067a4c283c00c7519e3: start","job_status":"start","queue_duration_s":1.634782,"scheduling_latency_s":1.634801} vm-test-run-nixbot-gitlab> gitlab # [ 229.370082] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.732Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"a6520067a4c283c00c7519e3","created_at":"2026-10-04T11:30:05.058Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-29d77d5f18744fbd","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"b20d2b0eadfc3af417057246635680a9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:05.074Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-a6520067a4c283c00c7519e3: done: 0.022997 sec","job_status":"done","queue_duration_s":1.634782,"scheduling_latency_s":1.634801,"redis_calls":15,"redis_duration_s":0.00183,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000586,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.001244,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":3,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":3,"duration_s":0.022997,"completed_at":"2026-10-04T11:30:06.732Z","load_balancing_strategy":"primary","db_duration_s":0.002741,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 229.413919] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.733Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"03eab725d622fdae24b4b66b","created_at":"2026-10-04T11:30:05.181Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-9c663bb3de6d4b11","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8f83b6976632dd2f964b9d48404bd010","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:05.191Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-03eab725d622fdae24b4b66b: start","job_status":"start","queue_duration_s":1.541799,"scheduling_latency_s":1.541818} vm-test-run-nixbot-gitlab> gitlab # [ 229.426945] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.744Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 229.428724] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.753Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"03eab725d622fdae24b4b66b","created_at":"2026-10-04T11:30:05.181Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-9c663bb3de6d4b11","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8f83b6976632dd2f964b9d48404bd010","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:05.191Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-03eab725d622fdae24b4b66b: done: 0.019588 sec","job_status":"done","queue_duration_s":1.541799,"scheduling_latency_s":1.541818,"redis_calls":10,"redis_duration_s":0.001412,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000598,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000814,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013847,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019588,"completed_at":"2026-10-04T11:30:06.753Z","load_balancing_strategy":"primary","db_duration_s":0.002255,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 229.469761] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.754Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c67d9f803793e93b03a33823","created_at":"2026-10-04T11:30:05.181Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-9c663bb3de6d4b11","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8f83b6976632dd2f964b9d48404bd010","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:05.191Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c67d9f803793e93b03a33823: start","job_status":"start","queue_duration_s":1.562541,"scheduling_latency_s":1.56256} vm-test-run-nixbot-gitlab> gitlab # [ 229.482917] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.764Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 229.484732] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.772Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c67d9f803793e93b03a33823","created_at":"2026-10-04T11:30:05.181Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-9c663bb3de6d4b11","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8f83b6976632dd2f964b9d48404bd010","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:05.191Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c67d9f803793e93b03a33823: done: 0.018012 sec","job_status":"done","queue_duration_s":1.562541,"scheduling_latency_s":1.56256,"redis_calls":10,"redis_duration_s":0.001381,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00059,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000791,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012757,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018012,"completed_at":"2026-10-04T11:30:06.772Z","load_balancing_strategy":"primary","db_duration_s":0.001524,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 229.524750] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.773Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cc3641cf50795d1aef43d13d","created_at":"2026-10-04T11:30:05.181Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-9c663bb3de6d4b11","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8f83b6976632dd2f964b9d48404bd010","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:05.191Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cc3641cf50795d1aef43d13d: start","job_status":"start","queue_duration_s":1.5817,"scheduling_latency_s":1.581719} vm-test-run-nixbot-gitlab> gitlab # [ 229.537808] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.783Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 229.539531] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.791Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cc3641cf50795d1aef43d13d","created_at":"2026-10-04T11:30:05.181Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-9c663bb3de6d4b11","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8f83b6976632dd2f964b9d48404bd010","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:05.191Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cc3641cf50795d1aef43d13d: done: 0.018003 sec","job_status":"done","queue_duration_s":1.5817,"scheduling_latency_s":1.581719,"redis_calls":10,"redis_duration_s":0.0014260000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000581,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000845,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012708,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018003,"completed_at":"2026-10-04T11:30:06.791Z","load_balancing_strategy":"primary","db_duration_s":0.001375,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 229.580426] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.792Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7f11991a32c5ada2b6c35d35","created_at":"2026-10-04T11:30:05.181Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-9c663bb3de6d4b11","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8f83b6976632dd2f964b9d48404bd010","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:05.191Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7f11991a32c5ada2b6c35d35: start","job_status":"start","queue_duration_s":1.600934,"scheduling_latency_s":1.600951} vm-test-run-nixbot-gitlab> gitlab # [ 229.593128] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.802Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 229.594859] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:30:06.810Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7f11991a32c5ada2b6c35d35","created_at":"2026-10-04T11:30:05.181Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-9c663bb3de6d4b11","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8f83b6976632dd2f964b9d48404bd010","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:30:05.191Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7f11991a32c5ada2b6c35d35: done: 0.017794 sec","job_status":"done","queue_duration_s":1.600934,"scheduling_latency_s":1.600951,"redis_calls":10,"redis_duration_s":0.001392,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000592,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.0008,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017794,"completed_at":"2026-10-04T11:30:06.810Z","load_balancing_strategy":"primary","db_duration_s":0.001375,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 251.961450] setup-gitlab-start[2834]: W, [2026-10-04T11:30:33.859749 #2834] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 279.447876] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:01.345Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"1e1f33a2a9dbbaaabea49960","created_at":"2026-10-04T11:31:01.298Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b4a8f0ac4151e176618762bd0aac666","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.343Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-1e1f33a2a9dbbaaabea49960: start","job_status":"start","queue_duration_s":0.002422,"scheduling_latency_s":0.002462} vm-test-run-nixbot-gitlab> gitlab # [ 279.635324] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:01.533Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"1e1f33a2a9dbbaaabea49960","created_at":"2026-10-04T11:31:01.298Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b4a8f0ac4151e176618762bd0aac666","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.343Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-1e1f33a2a9dbbaaabea49960: done: 0.188326 sec","job_status":"done","queue_duration_s":0.002422,"scheduling_latency_s":0.002462,"redis_calls":17,"redis_duration_s":0.023929,"redis_read_bytes":1285,"redis_write_bytes":1411,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.017261,"redis_feature_flag_read_bytes":1280,"redis_feature_flag_write_bytes":700,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005537,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001131,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029497,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.188326,"completed_at":"2026-10-04T11:31:01.533Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.057439,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 279.696309] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:01.539Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"398db67093babd0290174a6b","created_at":"2026-10-04T11:31:01.400Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.403Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-398db67093babd0290174a6b: start","job_status":"start","queue_duration_s":0.13536,"scheduling_latency_s":0.135426} vm-test-run-nixbot-gitlab> gitlab # [ 279.996501] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:01.894Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"398db67093babd0290174a6b","created_at":"2026-10-04T11:31:01.400Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.403Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-398db67093babd0290174a6b: done: 0.355674 sec","job_status":"done","queue_duration_s":0.13536,"scheduling_latency_s":0.135426,"redis_calls":65,"redis_duration_s":0.108484,"redis_read_bytes":748,"redis_write_bytes":41502,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.00453,"redis_feature_flag_read_bytes":640,"redis_feature_flag_write_bytes":393,"redis_queues_calls":2,"redis_queues_duration_s":0.002381,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.088462,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.013111,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.146349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.355674,"completed_at":"2026-10-04T11:31:01.894Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 280.062847] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:01.897Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"96eb5cfc0645070cf4d98f95","created_at":"2026-10-04T11:31:01.409Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"922c7784b6db8d23d0b3ba05f38bf3aa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T11:31:01.412Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-96eb5cfc0645070cf4d98f95: start","job_status":"start","queue_duration_s":0.484619,"scheduling_latency_s":0.484653} vm-test-run-nixbot-gitlab> gitlab # [ 280.079884] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:01.909Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 280.084864] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:01.916Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"96eb5cfc0645070cf4d98f95","created_at":"2026-10-04T11:31:01.409Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"922c7784b6db8d23d0b3ba05f38bf3aa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T11:31:01.412Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-96eb5cfc0645070cf4d98f95: done: 0.019639 sec","job_status":"done","queue_duration_s":0.484619,"scheduling_latency_s":0.484653,"redis_calls":8,"redis_duration_s":0.001675,"redis_read_bytes":133,"redis_write_bytes":914,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000187,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":108,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001039,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000449,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019639,"completed_at":"2026-10-04T11:31:01.916Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 280.131620] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:01.918Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"925027ac8278108f10832cbe","created_at":"2026-10-04T11:31:01.426Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a0b1e73a6ee77c294b135680b6e53a3","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.435Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-925027ac8278108f10832cbe: start","job_status":"start","queue_duration_s":0.483194,"scheduling_latency_s":0.483216} vm-test-run-nixbot-gitlab> gitlab # [ 280.146405] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:01.949Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"925027ac8278108f10832cbe","created_at":"2026-10-04T11:31:01.426Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a0b1e73a6ee77c294b135680b6e53a3","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.435Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-925027ac8278108f10832cbe: done: 0.03157 sec","job_status":"done","queue_duration_s":0.483194,"scheduling_latency_s":0.483216,"redis_calls":9,"redis_duration_s":0.0020570000000000002,"redis_read_bytes":261,"redis_write_bytes":985,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000371,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":159,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001208,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000478,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.03157,"completed_at":"2026-10-04T11:31:01.949Z","load_balancing_strategy":"primary","db_duration_s":0.009929,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 280.191473] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:01.951Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"980216379e5c87e5040bb920","created_at":"2026-10-04T11:31:01.483Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8c1fca0b3061ab06c06ec4ea89bf5c42","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.490Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-980216379e5c87e5040bb920: start","job_status":"start","queue_duration_s":0.461355,"scheduling_latency_s":0.46138} vm-test-run-nixbot-gitlab> gitlab # [ 280.206862] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:01.980Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"980216379e5c87e5040bb920","created_at":"2026-10-04T11:31:01.483Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8c1fca0b3061ab06c06ec4ea89bf5c42","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.490Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-980216379e5c87e5040bb920: done: 0.029168 sec","job_status":"done","queue_duration_s":0.461355,"scheduling_latency_s":0.46138,"redis_calls":11,"redis_duration_s":0.002455,"redis_read_bytes":136,"redis_write_bytes":1339,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000188,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":100,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001297,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019873,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0073404,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.029168,"completed_at":"2026-10-04T11:31:01.980Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003187,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 280.253411] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:01.982Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"15f68b2d141408ee7dddef09","created_at":"2026-10-04T11:31:01.505Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d47b522e1d1f3ca3df14b2f8865bcbd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.513Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-15f68b2d141408ee7dddef09: start","job_status":"start","queue_duration_s":0.469231,"scheduling_latency_s":0.469253} vm-test-run-nixbot-gitlab> gitlab # [ 280.268614] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.000Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"15f68b2d141408ee7dddef09","created_at":"2026-10-04T11:31:01.505Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d47b522e1d1f3ca3df14b2f8865bcbd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.513Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-15f68b2d141408ee7dddef09: done: 0.01865 sec","job_status":"done","queue_duration_s":0.469231,"scheduling_latency_s":0.469253,"redis_calls":10,"redis_duration_s":0.001984,"redis_read_bytes":389,"redis_write_bytes":1078,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000564,"redis_feature_flag_read_bytes":384,"redis_feature_flag_write_bytes":247,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001008,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000412,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01865,"completed_at":"2026-10-04T11:31:02.000Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 280.314651] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.002Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"eecd16d67fe148a45765fb1a","created_at":"2026-10-04T11:31:01.523Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1ea03088264d3db71853783fc74c9fe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T11:31:01.529Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-eecd16d67fe148a45765fb1a: start","job_status":"start","queue_duration_s":0.472729,"scheduling_latency_s":0.472747} vm-test-run-nixbot-gitlab> gitlab # [ 280.328724] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.011Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 280.332540] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.016Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"eecd16d67fe148a45765fb1a","created_at":"2026-10-04T11:31:01.523Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1ea03088264d3db71853783fc74c9fe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T11:31:01.529Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-eecd16d67fe148a45765fb1a: done: 0.014421 sec","job_status":"done","queue_duration_s":0.472729,"scheduling_latency_s":0.472747,"redis_calls":8,"redis_duration_s":0.0013729999999999999,"redis_read_bytes":133,"redis_write_bytes":908,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000155,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":107,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000337,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009293,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014421,"completed_at":"2026-10-04T11:31:02.016Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 280.375643] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.018Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"d6b64fd2166cd7f399605ba8","created_at":"2026-10-04T11:31:01.538Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e283959ef0d1c15899aa2f19ade6c20a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.543Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-d6b64fd2166cd7f399605ba8: start","job_status":"start","queue_duration_s":0.474249,"scheduling_latency_s":0.474267} vm-test-run-nixbot-gitlab> gitlab # [ 280.391120] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.058Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"d6b64fd2166cd7f399605ba8","created_at":"2026-10-04T11:31:01.538Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e283959ef0d1c15899aa2f19ade6c20a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.543Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-d6b64fd2166cd7f399605ba8: done: 0.040785 sec","job_status":"done","queue_duration_s":0.474249,"scheduling_latency_s":0.474267,"redis_calls":10,"redis_duration_s":0.002143,"redis_read_bytes":135,"redis_write_bytes":1275,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000152,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":106,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000997,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000994,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00070458,"exclusive_lock_hold_duration_s":0.01812182,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.040785,"completed_at":"2026-10-04T11:31:02.058Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015776,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 280.438048] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.060Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b393ef69560bc385bee75f8f","created_at":"2026-10-04T11:31:01.562Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a0992a4aaca5074361704c7ad66c042","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T11:31:01.572Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b393ef69560bc385bee75f8f: start","job_status":"start","queue_duration_s":0.488074,"scheduling_latency_s":0.488097} vm-test-run-nixbot-gitlab> gitlab # [ 280.452317] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.084Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b393ef69560bc385bee75f8f","created_at":"2026-10-04T11:31:01.562Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a0992a4aaca5074361704c7ad66c042","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T11:31:01.572Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b393ef69560bc385bee75f8f: done: 0.02415 sec","job_status":"done","queue_duration_s":0.488074,"scheduling_latency_s":0.488097,"redis_calls":8,"redis_duration_s":0.0016120000000000002,"redis_read_bytes":133,"redis_write_bytes":836,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000171,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":95,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001064,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000377,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02415,"completed_at":"2026-10-04T11:31:02.084Z","load_balancing_strategy":"primary","db_duration_s":0.001832,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 280.495409] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.086Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"100e74ff1c8e0e7c9b3d384a","created_at":"2026-10-04T11:31:01.586Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b05145b3c3d870ecf5299b943b0f53f4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.599Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-100e74ff1c8e0e7c9b3d384a: start","job_status":"start","queue_duration_s":0.486319,"scheduling_latency_s":0.486337} vm-test-run-nixbot-gitlab> gitlab # [ 280.510839] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.115Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"100e74ff1c8e0e7c9b3d384a","created_at":"2026-10-04T11:31:01.586Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b05145b3c3d870ecf5299b943b0f53f4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.599Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-100e74ff1c8e0e7c9b3d384a: done: 0.029611 sec","job_status":"done","queue_duration_s":0.486319,"scheduling_latency_s":0.486337,"redis_calls":16,"redis_duration_s":0.0032860000000000003,"redis_read_bytes":272,"redis_write_bytes":4229,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000354,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":199,"redis_queues_calls":2,"redis_queues_duration_s":0.000261,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001677,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000994,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029611,"completed_at":"2026-10-04T11:31:02.115Z","load_balancing_strategy":"primary","db_duration_s":0.001499,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 280.555701] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.116Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"27a8eee6ccf727c1093b1ff6","created_at":"2026-10-04T11:31:01.613Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8648808b907171843ed5d4664d21e535","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.620Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-27a8eee6ccf727c1093b1ff6: start","job_status":"start","queue_duration_s":0.496893,"scheduling_latency_s":0.49691} vm-test-run-nixbot-gitlab> gitlab # [ 280.570617] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.133Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"27a8eee6ccf727c1093b1ff6","created_at":"2026-10-04T11:31:01.613Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8648808b907171843ed5d4664d21e535","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.620Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-27a8eee6ccf727c1093b1ff6: done: 0.016823 sec","job_status":"done","queue_duration_s":0.496893,"scheduling_latency_s":0.49691,"redis_calls":10,"redis_duration_s":0.001838,"redis_read_bytes":135,"redis_write_bytes":1184,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000159,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":97,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000777,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010873,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00049406,"exclusive_lock_hold_duration_s":0.00286786,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.016823,"completed_at":"2026-10-04T11:31:02.133Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001186,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 280.616128] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.135Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"462962d0bf270b92d8d33021","created_at":"2026-10-04T11:31:01.636Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8fe808c9f419853d9d0fce25459566d","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.643Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-462962d0bf270b92d8d33021: start","job_status":"start","queue_duration_s":0.491257,"scheduling_latency_s":0.491273} vm-test-run-nixbot-gitlab> gitlab # [ 280.631013] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.150Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"462962d0bf270b92d8d33021","created_at":"2026-10-04T11:31:01.636Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8fe808c9f419853d9d0fce25459566d","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.643Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-462962d0bf270b92d8d33021: done: 0.015738 sec","job_status":"done","queue_duration_s":0.491257,"scheduling_latency_s":0.491273,"redis_calls":8,"redis_duration_s":0.001478,"redis_read_bytes":133,"redis_write_bytes":764,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000162,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":83,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000957,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000359,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015738,"completed_at":"2026-10-04T11:31:02.150Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001076,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 280.674874] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.152Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"f50bd6d7fda4c13f0ce9800e","created_at":"2026-10-04T11:31:01.673Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cf93113d3d675ae87db43e6cf26a5c2c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.680Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-f50bd6d7fda4c13f0ce9800e: start","job_status":"start","queue_duration_s":0.471879,"scheduling_latency_s":0.471896} vm-test-run-nixbot-gitlab> gitlab # [ 280.690243] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.159Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 280.693707] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.165Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"f50bd6d7fda4c13f0ce9800e","created_at":"2026-10-04T11:31:01.673Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cf93113d3d675ae87db43e6cf26a5c2c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.680Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-f50bd6d7fda4c13f0ce9800e: done: 0.012979 sec","job_status":"done","queue_duration_s":0.471879,"scheduling_latency_s":0.471896,"redis_calls":8,"redis_duration_s":0.001368,"redis_read_bytes":133,"redis_write_bytes":873,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000153,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":102,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000336,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012979,"completed_at":"2026-10-04T11:31:02.164Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 280.739135] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.166Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"4444d49c8f0f93e79e8213e8","created_at":"2026-10-04T11:31:01.693Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2e70e9a8b4bf3a96e45b9e0f2115876","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.701Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-4444d49c8f0f93e79e8213e8: start","job_status":"start","queue_duration_s":0.464825,"scheduling_latency_s":0.464843} vm-test-run-nixbot-gitlab> gitlab # [ 280.753624] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.183Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"4444d49c8f0f93e79e8213e8","created_at":"2026-10-04T11:31:01.693Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2e70e9a8b4bf3a96e45b9e0f2115876","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.701Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-4444d49c8f0f93e79e8213e8: done: 0.017201 sec","job_status":"done","queue_duration_s":0.464825,"scheduling_latency_s":0.464843,"redis_calls":10,"redis_duration_s":0.001877,"redis_read_bytes":135,"redis_write_bytes":1167,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000151,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":95,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000823,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011249,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00049852,"exclusive_lock_hold_duration_s":0.00315972,"duration_s":0.017201,"completed_at":"2026-10-04T11:31:02.183Z","load_balancing_strategy":"primary","db_duration_s":0.001049,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 280.796915] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.188Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"becc8c86e05aedd0111bf7a8","created_at":"2026-10-04T11:31:01.718Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e4a55a1b189c7f9db99931b2bf5b556","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.725Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-becc8c86e05aedd0111bf7a8: start","job_status":"start","queue_duration_s":0.462843,"scheduling_latency_s":0.462865} vm-test-run-nixbot-gitlab> gitlab # [ 280.811805] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.199Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 280.815773] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.204Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"becc8c86e05aedd0111bf7a8","created_at":"2026-10-04T11:31:01.718Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e4a55a1b189c7f9db99931b2bf5b556","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.725Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-becc8c86e05aedd0111bf7a8: done: 0.016574 sec","job_status":"done","queue_duration_s":0.462843,"scheduling_latency_s":0.462865,"redis_calls":8,"redis_duration_s":0.002551,"redis_read_bytes":133,"redis_write_bytes":867,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.001351,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":101,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00087,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00033,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016574,"completed_at":"2026-10-04T11:31:02.204Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 280.860376] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.206Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"79de029b0ebaf2381b21c131","created_at":"2026-10-04T11:31:01.743Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f54373ccc870df30d079174ff07c6ad2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.752Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-79de029b0ebaf2381b21c131: start","job_status":"start","queue_duration_s":0.453792,"scheduling_latency_s":0.453811} vm-test-run-nixbot-gitlab> gitlab # [ 280.875194] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.437Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"79de029b0ebaf2381b21c131","created_at":"2026-10-04T11:31:01.743Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f54373ccc870df30d079174ff07c6ad2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.752Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-79de029b0ebaf2381b21c131: done: 0.231651 sec","job_status":"done","queue_duration_s":0.453792,"scheduling_latency_s":0.453811,"redis_calls":11,"redis_duration_s":0.001744,"redis_read_bytes":263,"redis_write_bytes":1138,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000248,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":151,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000755,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.03,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.19437,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00050084,"exclusive_lock_hold_duration_s":0.21609064,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.231651,"completed_at":"2026-10-04T11:31:02.437Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.04501,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 280.922813] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.439Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"bce98eabfa918fe7e41c59c2","created_at":"2026-10-04T11:31:01.763Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2bf5810cb9d2d83572c8c8482f4fd07d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.770Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-bce98eabfa918fe7e41c59c2: start","job_status":"start","queue_duration_s":0.668483,"scheduling_latency_s":0.668504} vm-test-run-nixbot-gitlab> gitlab # [ 280.938312] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.453Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"bce98eabfa918fe7e41c59c2","created_at":"2026-10-04T11:31:01.763Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2bf5810cb9d2d83572c8c8482f4fd07d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.770Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-bce98eabfa918fe7e41c59c2: done: 0.014867 sec","job_status":"done","queue_duration_s":0.668483,"scheduling_latency_s":0.668504,"redis_calls":8,"redis_duration_s":0.001254,"redis_read_bytes":133,"redis_write_bytes":879,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000164,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":103,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000781,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000309,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014867,"completed_at":"2026-10-04T11:31:02.453Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 280.982196] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.455Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"bc49f2b3ece4696adaddbe15","created_at":"2026-10-04T11:31:01.784Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1ebbaa092c7e754119044ee941e42a1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.792Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-bc49f2b3ece4696adaddbe15: start","job_status":"start","queue_duration_s":0.663059,"scheduling_latency_s":0.663077} vm-test-run-nixbot-gitlab> gitlab # [ 280.997350] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.491Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"bc49f2b3ece4696adaddbe15","created_at":"2026-10-04T11:31:01.784Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1ebbaa092c7e754119044ee941e42a1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.792Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-bc49f2b3ece4696adaddbe15: done: 0.036259 sec","job_status":"done","queue_duration_s":0.663059,"scheduling_latency_s":0.663077,"redis_calls":10,"redis_duration_s":0.001608,"redis_read_bytes":135,"redis_write_bytes":1267,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.00012,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":105,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000784,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000704,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00051408,"exclusive_lock_hold_duration_s":0.01941632,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.036259,"completed_at":"2026-10-04T11:31:02.491Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010419,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 281.043813] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.492Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"16ff3c60e9180ff75d620646","created_at":"2026-10-04T11:31:01.819Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be2cfa86b3b9fa801d179e65fc10cfa7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.826Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-16ff3c60e9180ff75d620646: start","job_status":"start","queue_duration_s":0.666282,"scheduling_latency_s":0.666301} vm-test-run-nixbot-gitlab> gitlab # [ 281.059357] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.502Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 281.063030] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.507Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"16ff3c60e9180ff75d620646","created_at":"2026-10-04T11:31:01.819Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be2cfa86b3b9fa801d179e65fc10cfa7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.826Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-16ff3c60e9180ff75d620646: done: 0.014742 sec","job_status":"done","queue_duration_s":0.666282,"scheduling_latency_s":0.666301,"redis_calls":8,"redis_duration_s":0.0012330000000000002,"redis_read_bytes":133,"redis_write_bytes":937,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000131,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":111,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000796,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000306,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014742,"completed_at":"2026-10-04T11:31:02.507Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 281.107729] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.508Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0972de844aba01a617e53b0a","created_at":"2026-10-04T11:31:01.847Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2799fdf34e08da70c884e81773ce58ec","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.856Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0972de844aba01a617e53b0a: start","job_status":"start","queue_duration_s":0.652236,"scheduling_latency_s":0.652254} vm-test-run-nixbot-gitlab> gitlab # [ 281.123938] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.847Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0972de844aba01a617e53b0a","created_at":"2026-10-04T11:31:01.847Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2799fdf34e08da70c884e81773ce58ec","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.856Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0972de844aba01a617e53b0a: done: 0.338204 sec","job_status":"done","queue_duration_s":0.652236,"scheduling_latency_s":0.652254,"redis_calls":562,"redis_duration_s":0.064733,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.063762,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.310573,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.338204,"completed_at":"2026-10-04T11:31:02.847Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 281.166217] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.848Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b9b6ce98ec7207b4d08c6fd","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b9b6ce98ec7207b4d08c6fd: start","job_status":"start","queue_duration_s":0.97704,"scheduling_latency_s":0.977066} vm-test-run-nixbot-gitlab> gitlab # [ 281.179113] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b9b6ce98ec7207b4d08c6fd","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b9b6ce98ec7207b4d08c6fd: done: 0.017558 sec","job_status":"done","queue_duration_s":0.97704,"scheduling_latency_s":0.977066,"redis_calls":14,"redis_duration_s":0.0018009999999999999,"redis_read_bytes":138,"redis_write_bytes":1242,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000167,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":85,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012294,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017558,"completed_at":"2026-10-04T11:31:02.866Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 281.222005] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d4290263b4cbfdb09f2f498","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d4290263b4cbfdb09f2f498: start","job_status":"start","queue_duration_s":0.995774,"scheduling_latency_s":0.995793} vm-test-run-nixbot-gitlab> gitlab # [ 281.235016] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.883Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d4290263b4cbfdb09f2f498","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d4290263b4cbfdb09f2f498: done: 0.015793 sec","job_status":"done","queue_duration_s":0.995774,"scheduling_latency_s":0.995793,"redis_calls":13,"redis_duration_s":0.001709,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000673,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015793,"completed_at":"2026-10-04T11:31:02.883Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 281.275508] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.885Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"059ec962b20cbe09d2791707","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-059ec962b20cbe09d2791707: start","job_status":"start","queue_duration_s":1.013771,"scheduling_latency_s":1.013787} vm-test-run-nixbot-gitlab> gitlab # [ 281.288499] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.901Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"059ec962b20cbe09d2791707","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-059ec962b20cbe09d2791707: done: 0.016525 sec","job_status":"done","queue_duration_s":1.013771,"scheduling_latency_s":1.013787,"redis_calls":13,"redis_duration_s":0.002365,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000713,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001652,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011252,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016525,"completed_at":"2026-10-04T11:31:02.901Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 281.330467] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e71b490d79530f5d7601e144","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e71b490d79530f5d7601e144: start","job_status":"start","queue_duration_s":1.031382,"scheduling_latency_s":1.031398} vm-test-run-nixbot-gitlab> gitlab # [ 281.343472] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e71b490d79530f5d7601e144","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e71b490d79530f5d7601e144: done: 0.015683 sec","job_status":"done","queue_duration_s":1.031382,"scheduling_latency_s":1.031398,"redis_calls":13,"redis_duration_s":0.001967,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000703,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001264,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015683,"completed_at":"2026-10-04T11:31:02.918Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 281.384175] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5af69ab4de02311af2d609f","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5af69ab4de02311af2d609f: start","job_status":"start","queue_duration_s":1.048272,"scheduling_latency_s":1.048288} vm-test-run-nixbot-gitlab> gitlab # [ 281.397117] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5af69ab4de02311af2d609f","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5af69ab4de02311af2d609f: done: 0.015076 sec","job_status":"done","queue_duration_s":1.048272,"scheduling_latency_s":1.048288,"redis_calls":13,"redis_duration_s":0.001978,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000664,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001314,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009417,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015076,"completed_at":"2026-10-04T11:31:02.934Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 281.437851] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.937Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"275c6780b70de5d457ced5e6","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-275c6780b70de5d457ced5e6: start","job_status":"start","queue_duration_s":1.065405,"scheduling_latency_s":1.065422} vm-test-run-nixbot-gitlab> gitlab # [ 281.450536] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.952Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"275c6780b70de5d457ced5e6","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-275c6780b70de5d457ced5e6: done: 0.015168 sec","job_status":"done","queue_duration_s":1.065405,"scheduling_latency_s":1.065422,"redis_calls":13,"redis_duration_s":0.0018800000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000679,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001201,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015168,"completed_at":"2026-10-04T11:31:02.952Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 281.491685] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.953Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"771833757445055809883b9c","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-771833757445055809883b9c: start","job_status":"start","queue_duration_s":1.081737,"scheduling_latency_s":1.081752} vm-test-run-nixbot-gitlab> gitlab # [ 281.504907] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.968Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"771833757445055809883b9c","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-771833757445055809883b9c: done: 0.015525 sec","job_status":"done","queue_duration_s":1.081737,"scheduling_latency_s":1.081752,"redis_calls":13,"redis_duration_s":0.002007,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000675,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001332,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015525,"completed_at":"2026-10-04T11:31:02.968Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 281.547067] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"94e10021dc644ff9ea7051b5","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-94e10021dc644ff9ea7051b5: start","job_status":"start","queue_duration_s":1.098407,"scheduling_latency_s":1.098423} vm-test-run-nixbot-gitlab> gitlab # [ 281.560276] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.984Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"94e10021dc644ff9ea7051b5","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-94e10021dc644ff9ea7051b5: done: 0.014829 sec","job_status":"done","queue_duration_s":1.098407,"scheduling_latency_s":1.098423,"redis_calls":13,"redis_duration_s":0.001901,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000643,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001258,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014829,"completed_at":"2026-10-04T11:31:02.984Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 281.601197] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:02.986Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbb5e1c1c1908d80b66e272f","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbb5e1c1c1908d80b66e272f: start","job_status":"start","queue_duration_s":1.115167,"scheduling_latency_s":1.115184} vm-test-run-nixbot-gitlab> gitlab # [ 281.614008] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.002Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbb5e1c1c1908d80b66e272f","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbb5e1c1c1908d80b66e272f: done: 0.015491 sec","job_status":"done","queue_duration_s":1.115167,"scheduling_latency_s":1.115184,"redis_calls":13,"redis_duration_s":0.002247,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000684,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001563,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015491,"completed_at":"2026-10-04T11:31:03.002Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 281.654239] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.003Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b5f12e917a92d6748806789","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b5f12e917a92d6748806789: start","job_status":"start","queue_duration_s":1.131872,"scheduling_latency_s":1.131889} vm-test-run-nixbot-gitlab> gitlab # [ 281.667213] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.018Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b5f12e917a92d6748806789","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b5f12e917a92d6748806789: done: 0.015308 sec","job_status":"done","queue_duration_s":1.131872,"scheduling_latency_s":1.131889,"redis_calls":13,"redis_duration_s":0.002111,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00137,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015308,"completed_at":"2026-10-04T11:31:03.018Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 281.708217] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04c0cfbfda335ea93eb7159a","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04c0cfbfda335ea93eb7159a: start","job_status":"start","queue_duration_s":1.1485,"scheduling_latency_s":1.148516} vm-test-run-nixbot-gitlab> gitlab # [ 281.721998] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.034Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04c0cfbfda335ea93eb7159a","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04c0cfbfda335ea93eb7159a: done: 0.014767 sec","job_status":"done","queue_duration_s":1.1485,"scheduling_latency_s":1.148516,"redis_calls":13,"redis_duration_s":0.001899,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000652,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001247,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009169,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014767,"completed_at":"2026-10-04T11:31:03.034Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 281.762292] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f73a23e275f193e1f5c96312","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f73a23e275f193e1f5c96312: start","job_status":"start","queue_duration_s":1.165272,"scheduling_latency_s":1.165288} vm-test-run-nixbot-gitlab> gitlab # [ 281.775168] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f73a23e275f193e1f5c96312","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f73a23e275f193e1f5c96312: done: 0.019957 sec","job_status":"done","queue_duration_s":1.165272,"scheduling_latency_s":1.165288,"redis_calls":13,"redis_duration_s":0.002547,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001246,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001301,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019957,"completed_at":"2026-10-04T11:31:03.056Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 281.816150] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.058Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f2b5df38d4dc39f7ed6d95f","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f2b5df38d4dc39f7ed6d95f: start","job_status":"start","queue_duration_s":1.186804,"scheduling_latency_s":1.186826} vm-test-run-nixbot-gitlab> gitlab # [ 281.828919] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.079Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f2b5df38d4dc39f7ed6d95f","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f2b5df38d4dc39f7ed6d95f: done: 0.021007 sec","job_status":"done","queue_duration_s":1.186804,"scheduling_latency_s":1.186826,"redis_calls":13,"redis_duration_s":0.002091,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000711,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00138,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021007,"completed_at":"2026-10-04T11:31:03.079Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 281.869112] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.080Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be9fc32285da05022fe6acb0","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be9fc32285da05022fe6acb0: start","job_status":"start","queue_duration_s":1.209308,"scheduling_latency_s":1.209329} vm-test-run-nixbot-gitlab> gitlab # [ 281.881894] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.102Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be9fc32285da05022fe6acb0","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be9fc32285da05022fe6acb0: done: 0.021698 sec","job_status":"done","queue_duration_s":1.209308,"scheduling_latency_s":1.209329,"redis_calls":13,"redis_duration_s":0.0026490000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001858,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021698,"completed_at":"2026-10-04T11:31:03.102Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 281.922891] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.105Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43566cc4ea0af857b2a2c2d1","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43566cc4ea0af857b2a2c2d1: start","job_status":"start","queue_duration_s":1.233884,"scheduling_latency_s":1.233908} vm-test-run-nixbot-gitlab> gitlab # [ 281.937238] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43566cc4ea0af857b2a2c2d1","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43566cc4ea0af857b2a2c2d1: done: 0.021061 sec","job_status":"done","queue_duration_s":1.233884,"scheduling_latency_s":1.233908,"redis_calls":13,"redis_duration_s":0.0023179999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000794,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001524,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021061,"completed_at":"2026-10-04T11:31:03.126Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 281.977952] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"42be3978784f7a87566e68ed","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-42be3978784f7a87566e68ed: start","job_status":"start","queue_duration_s":1.25635,"scheduling_latency_s":1.256368} vm-test-run-nixbot-gitlab> gitlab # [ 281.990785] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.145Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"42be3978784f7a87566e68ed","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-42be3978784f7a87566e68ed: done: 0.017432 sec","job_status":"done","queue_duration_s":1.25635,"scheduling_latency_s":1.256368,"redis_calls":13,"redis_duration_s":0.0020759999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000688,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001388,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017432,"completed_at":"2026-10-04T11:31:03.145Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 282.031614] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bb5ad7eb0a3e7685dc884d7","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bb5ad7eb0a3e7685dc884d7: start","job_status":"start","queue_duration_s":1.275013,"scheduling_latency_s":1.275031} vm-test-run-nixbot-gitlab> gitlab # [ 282.044720] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.162Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bb5ad7eb0a3e7685dc884d7","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bb5ad7eb0a3e7685dc884d7: done: 0.015978 sec","job_status":"done","queue_duration_s":1.275013,"scheduling_latency_s":1.275031,"redis_calls":13,"redis_duration_s":0.002944,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000733,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002211,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015978,"completed_at":"2026-10-04T11:31:03.162Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 282.085363] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.163Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9ef09522d31464f93cba7a7","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9ef09522d31464f93cba7a7: start","job_status":"start","queue_duration_s":1.292305,"scheduling_latency_s":1.292323} vm-test-run-nixbot-gitlab> gitlab # [ 282.098307] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.179Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9ef09522d31464f93cba7a7","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9ef09522d31464f93cba7a7: done: 0.015125 sec","job_status":"done","queue_duration_s":1.292305,"scheduling_latency_s":1.292323,"redis_calls":13,"redis_duration_s":0.001888,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000648,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00124,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015125,"completed_at":"2026-10-04T11:31:03.178Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 282.140406] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.180Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a8d1640e977feb30bdb5804","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a8d1640e977feb30bdb5804: start","job_status":"start","queue_duration_s":1.309125,"scheduling_latency_s":1.309147} vm-test-run-nixbot-gitlab> gitlab # [ 282.153263] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.195Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a8d1640e977feb30bdb5804","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a8d1640e977feb30bdb5804: done: 0.014861 sec","job_status":"done","queue_duration_s":1.309125,"scheduling_latency_s":1.309147,"redis_calls":13,"redis_duration_s":0.001705,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00061,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014861,"completed_at":"2026-10-04T11:31:03.195Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 282.194665] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.196Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1961590050201f389e64888","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1961590050201f389e64888: start","job_status":"start","queue_duration_s":1.325161,"scheduling_latency_s":1.325177} vm-test-run-nixbot-gitlab> gitlab # [ 282.207548] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.213Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1961590050201f389e64888","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1961590050201f389e64888: done: 0.016475 sec","job_status":"done","queue_duration_s":1.325161,"scheduling_latency_s":1.325177,"redis_calls":13,"redis_duration_s":0.0017920000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000671,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001121,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011137,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016475,"completed_at":"2026-10-04T11:31:03.213Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 282.248766] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.214Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e838cf20cfdf1bfbc900a830","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e838cf20cfdf1bfbc900a830: start","job_status":"start","queue_duration_s":1.342979,"scheduling_latency_s":1.342997} vm-test-run-nixbot-gitlab> gitlab # [ 282.261813] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.234Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e838cf20cfdf1bfbc900a830","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e838cf20cfdf1bfbc900a830: done: 0.020463 sec","job_status":"done","queue_duration_s":1.342979,"scheduling_latency_s":1.342997,"redis_calls":13,"redis_duration_s":0.001772,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000632,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00114,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020463,"completed_at":"2026-10-04T11:31:03.234Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 282.302869] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75833b4757b7def2529bf2c3","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75833b4757b7def2529bf2c3: start","job_status":"start","queue_duration_s":1.364984,"scheduling_latency_s":1.365001} vm-test-run-nixbot-gitlab> gitlab # [ 282.316162] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.251Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75833b4757b7def2529bf2c3","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75833b4757b7def2529bf2c3: done: 0.014936 sec","job_status":"done","queue_duration_s":1.364984,"scheduling_latency_s":1.365001,"redis_calls":13,"redis_duration_s":0.001565,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010473,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014936,"completed_at":"2026-10-04T11:31:03.251Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 282.357786] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7dbda056114234e475a5a21f","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7dbda056114234e475a5a21f: start","job_status":"start","queue_duration_s":1.38106,"scheduling_latency_s":1.381077} vm-test-run-nixbot-gitlab> gitlab # [ 282.370920] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7dbda056114234e475a5a21f","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7dbda056114234e475a5a21f: done: 0.0149 sec","job_status":"done","queue_duration_s":1.38106,"scheduling_latency_s":1.381077,"redis_calls":13,"redis_duration_s":0.001586,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010469,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0149,"completed_at":"2026-10-04T11:31:03.267Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 282.412165] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.268Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3efe3a0cc5d43481dceeb6a","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3efe3a0cc5d43481dceeb6a: start","job_status":"start","queue_duration_s":1.397185,"scheduling_latency_s":1.397203} vm-test-run-nixbot-gitlab> gitlab # [ 282.424933] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3efe3a0cc5d43481dceeb6a","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3efe3a0cc5d43481dceeb6a: done: 0.015168 sec","job_status":"done","queue_duration_s":1.397185,"scheduling_latency_s":1.397203,"redis_calls":13,"redis_duration_s":0.0017009999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001145,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009816,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015168,"completed_at":"2026-10-04T11:31:03.283Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 282.465435] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc989d0fa0420ac0ba0e371a","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc989d0fa0420ac0ba0e371a: start","job_status":"start","queue_duration_s":1.414074,"scheduling_latency_s":1.41409} vm-test-run-nixbot-gitlab> gitlab # [ 282.478340] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.301Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc989d0fa0420ac0ba0e371a","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc989d0fa0420ac0ba0e371a: done: 0.015508 sec","job_status":"done","queue_duration_s":1.414074,"scheduling_latency_s":1.41409,"redis_calls":13,"redis_duration_s":0.001599,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000633,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010711,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015508,"completed_at":"2026-10-04T11:31:03.301Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 282.519718] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58a14a8322b35fc28e8eff15","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58a14a8322b35fc28e8eff15: start","job_status":"start","queue_duration_s":1.430656,"scheduling_latency_s":1.430673} vm-test-run-nixbot-gitlab> gitlab # [ 282.533069] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.317Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58a14a8322b35fc28e8eff15","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58a14a8322b35fc28e8eff15: done: 0.014983 sec","job_status":"done","queue_duration_s":1.430656,"scheduling_latency_s":1.430673,"redis_calls":13,"redis_duration_s":0.0015599999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010453,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014983,"completed_at":"2026-10-04T11:31:03.317Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 282.575770] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6ce6b0e170199bfdfd1d874","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6ce6b0e170199bfdfd1d874: start","job_status":"start","queue_duration_s":1.446768,"scheduling_latency_s":1.446785} vm-test-run-nixbot-gitlab> gitlab # [ 282.588760] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6ce6b0e170199bfdfd1d874","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6ce6b0e170199bfdfd1d874: done: 0.01523 sec","job_status":"done","queue_duration_s":1.446768,"scheduling_latency_s":1.446785,"redis_calls":13,"redis_duration_s":0.0018700000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000577,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001293,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010559,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01523,"completed_at":"2026-10-04T11:31:03.333Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 282.631238] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af9949b11101de619d9d168f","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af9949b11101de619d9d168f: start","job_status":"start","queue_duration_s":1.463127,"scheduling_latency_s":1.463144} vm-test-run-nixbot-gitlab> gitlab # [ 282.644629] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af9949b11101de619d9d168f","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af9949b11101de619d9d168f: done: 0.014659 sec","job_status":"done","queue_duration_s":1.463127,"scheduling_latency_s":1.463144,"redis_calls":13,"redis_duration_s":0.001491,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014659,"completed_at":"2026-10-04T11:31:03.349Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 282.685321] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.351Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5554456746c8dc0c72aa17f","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5554456746c8dc0c72aa17f: start","job_status":"start","queue_duration_s":1.479494,"scheduling_latency_s":1.47951} vm-test-run-nixbot-gitlab> gitlab # [ 282.698180] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5554456746c8dc0c72aa17f","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5554456746c8dc0c72aa17f: done: 0.014858 sec","job_status":"done","queue_duration_s":1.479494,"scheduling_latency_s":1.47951,"redis_calls":13,"redis_duration_s":0.0015040000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010445,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014858,"completed_at":"2026-10-04T11:31:03.365Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 282.739145] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.367Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c952f6a355b3acce4606b11","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c952f6a355b3acce4606b11: start","job_status":"start","queue_duration_s":1.495485,"scheduling_latency_s":1.495503} vm-test-run-nixbot-gitlab> gitlab # [ 282.753503] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.381Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c952f6a355b3acce4606b11","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c952f6a355b3acce4606b11: done: 0.014912 sec","job_status":"done","queue_duration_s":1.495485,"scheduling_latency_s":1.495503,"redis_calls":13,"redis_duration_s":0.0021650000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001191,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010502,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014912,"completed_at":"2026-10-04T11:31:03.381Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 282.794404] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.383Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ac355abd2c9a8432918a1c5","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ac355abd2c9a8432918a1c5: start","job_status":"start","queue_duration_s":1.511613,"scheduling_latency_s":1.51163} vm-test-run-nixbot-gitlab> gitlab # [ 282.807319] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ac355abd2c9a8432918a1c5","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ac355abd2c9a8432918a1c5: done: 0.014915 sec","job_status":"done","queue_duration_s":1.511613,"scheduling_latency_s":1.51163,"redis_calls":13,"redis_duration_s":0.0015240000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001007,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009506,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014915,"completed_at":"2026-10-04T11:31:03.398Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 282.848366] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f27198e2fbf9e3509c019d9e","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f27198e2fbf9e3509c019d9e: start","job_status":"start","queue_duration_s":1.528206,"scheduling_latency_s":1.528223} vm-test-run-nixbot-gitlab> gitlab # [ 282.861182] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.414Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f27198e2fbf9e3509c019d9e","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f27198e2fbf9e3509c019d9e: done: 0.014608 sec","job_status":"done","queue_duration_s":1.528206,"scheduling_latency_s":1.528223,"redis_calls":13,"redis_duration_s":0.001562,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00101,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014608,"completed_at":"2026-10-04T11:31:03.414Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 282.901807] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.415Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"498f5d9375812db494b70771","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-498f5d9375812db494b70771: start","job_status":"start","queue_duration_s":1.543968,"scheduling_latency_s":1.543986} vm-test-run-nixbot-gitlab> gitlab # [ 282.914817] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"498f5d9375812db494b70771","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-498f5d9375812db494b70771: done: 0.016443 sec","job_status":"done","queue_duration_s":1.543968,"scheduling_latency_s":1.543986,"redis_calls":13,"redis_duration_s":0.0017009999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000589,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001112,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016443,"completed_at":"2026-10-04T11:31:03.431Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 282.956922] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"423cce040dda2496f1033a7d","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-423cce040dda2496f1033a7d: start","job_status":"start","queue_duration_s":1.561496,"scheduling_latency_s":1.561512} vm-test-run-nixbot-gitlab> gitlab # [ 282.969728] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.447Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"423cce040dda2496f1033a7d","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-423cce040dda2496f1033a7d: done: 0.014288 sec","job_status":"done","queue_duration_s":1.561496,"scheduling_latency_s":1.561512,"redis_calls":13,"redis_duration_s":0.001507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014288,"completed_at":"2026-10-04T11:31:03.447Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 283.010105] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.448Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7117a085fc3dd6c5ed8067de","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7117a085fc3dd6c5ed8067de: start","job_status":"start","queue_duration_s":1.577342,"scheduling_latency_s":1.577359} vm-test-run-nixbot-gitlab> gitlab # [ 283.022904] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.463Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7117a085fc3dd6c5ed8067de","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7117a085fc3dd6c5ed8067de: done: 0.014395 sec","job_status":"done","queue_duration_s":1.577342,"scheduling_latency_s":1.577359,"redis_calls":13,"redis_duration_s":0.001483,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014395,"completed_at":"2026-10-04T11:31:03.463Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 283.063849] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41294880a7caf11d4e648cd3","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41294880a7caf11d4e648cd3: start","job_status":"start","queue_duration_s":1.592888,"scheduling_latency_s":1.592906} vm-test-run-nixbot-gitlab> gitlab # [ 283.076827] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.478Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41294880a7caf11d4e648cd3","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41294880a7caf11d4e648cd3: done: 0.014034 sec","job_status":"done","queue_duration_s":1.592888,"scheduling_latency_s":1.592906,"redis_calls":13,"redis_duration_s":0.00148,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014034,"completed_at":"2026-10-04T11:31:03.478Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 283.118362] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.479Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4be3ad8a7ab582a9150341f2","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4be3ad8a7ab582a9150341f2: start","job_status":"start","queue_duration_s":1.60814,"scheduling_latency_s":1.608171} vm-test-run-nixbot-gitlab> gitlab # [ 283.131780] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.493Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4be3ad8a7ab582a9150341f2","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4be3ad8a7ab582a9150341f2: done: 0.014051 sec","job_status":"done","queue_duration_s":1.60814,"scheduling_latency_s":1.608171,"redis_calls":13,"redis_duration_s":0.001496,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014051,"completed_at":"2026-10-04T11:31:03.493Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 283.174137] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.494Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64da98460ded78c24c7f1288","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64da98460ded78c24c7f1288: start","job_status":"start","queue_duration_s":1.623279,"scheduling_latency_s":1.623295} vm-test-run-nixbot-gitlab> gitlab # [ 283.194463] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.509Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64da98460ded78c24c7f1288","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64da98460ded78c24c7f1288: done: 0.014784 sec","job_status":"done","queue_duration_s":1.623279,"scheduling_latency_s":1.623295,"redis_calls":13,"redis_duration_s":0.00161,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000577,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014784,"completed_at":"2026-10-04T11:31:03.509Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 283.250383] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5246ccc2600ae85a4e47ba2d","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5246ccc2600ae85a4e47ba2d: start","job_status":"start","queue_duration_s":1.639155,"scheduling_latency_s":1.639171} vm-test-run-nixbot-gitlab> gitlab # [ 283.264377] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5246ccc2600ae85a4e47ba2d","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5246ccc2600ae85a4e47ba2d: done: 0.014431 sec","job_status":"done","queue_duration_s":1.639155,"scheduling_latency_s":1.639171,"redis_calls":13,"redis_duration_s":0.001496,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009669,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014431,"completed_at":"2026-10-04T11:31:03.525Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 283.305219] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.526Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b7779d97ef724dd29b72c4a","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b7779d97ef724dd29b72c4a: start","job_status":"start","queue_duration_s":1.65482,"scheduling_latency_s":1.654837} vm-test-run-nixbot-gitlab> gitlab # [ 283.318059] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b7779d97ef724dd29b72c4a","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b7779d97ef724dd29b72c4a: done: 0.014424 sec","job_status":"done","queue_duration_s":1.65482,"scheduling_latency_s":1.654837,"redis_calls":13,"redis_duration_s":0.001503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009706,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014424,"completed_at":"2026-10-04T11:31:03.540Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 283.358656] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.542Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0051da4788cf093ce059ce97","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0051da4788cf093ce059ce97: start","job_status":"start","queue_duration_s":1.670378,"scheduling_latency_s":1.670394} vm-test-run-nixbot-gitlab> gitlab # [ 283.372606] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0051da4788cf093ce059ce97","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0051da4788cf093ce059ce97: done: 0.015011 sec","job_status":"done","queue_duration_s":1.670378,"scheduling_latency_s":1.670394,"redis_calls":13,"redis_duration_s":0.0015769999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000568,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009969,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015011,"completed_at":"2026-10-04T11:31:03.556Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 283.413788] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a018009888bb1bf69c8a235b","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a018009888bb1bf69c8a235b: start","job_status":"start","queue_duration_s":1.68651,"scheduling_latency_s":1.686528} vm-test-run-nixbot-gitlab> gitlab # [ 283.426995] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a018009888bb1bf69c8a235b","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a018009888bb1bf69c8a235b: done: 0.014149 sec","job_status":"done","queue_duration_s":1.68651,"scheduling_latency_s":1.686528,"redis_calls":13,"redis_duration_s":0.001493,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000962,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014149,"completed_at":"2026-10-04T11:31:03.572Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 283.469431] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.573Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04bf3b9bdd73af80b4618fbd","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04bf3b9bdd73af80b4618fbd: start","job_status":"start","queue_duration_s":1.701756,"scheduling_latency_s":1.701774} vm-test-run-nixbot-gitlab> gitlab # [ 283.482616] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.587Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04bf3b9bdd73af80b4618fbd","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04bf3b9bdd73af80b4618fbd: done: 0.014073 sec","job_status":"done","queue_duration_s":1.701756,"scheduling_latency_s":1.701774,"redis_calls":13,"redis_duration_s":0.001485,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009672,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014073,"completed_at":"2026-10-04T11:31:03.587Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 283.523672] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.588Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ca4fbbfbc9d596e7c289f07","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ca4fbbfbc9d596e7c289f07: start","job_status":"start","queue_duration_s":1.716949,"scheduling_latency_s":1.716965} vm-test-run-nixbot-gitlab> gitlab # [ 283.536894] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.602Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ca4fbbfbc9d596e7c289f07","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ca4fbbfbc9d596e7c289f07: done: 0.014433 sec","job_status":"done","queue_duration_s":1.716949,"scheduling_latency_s":1.716965,"redis_calls":13,"redis_duration_s":0.0018080000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000838,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014433,"completed_at":"2026-10-04T11:31:03.602Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 283.578847] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.604Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f157fef9d471ae9c4db0ae77","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f157fef9d471ae9c4db0ae77: start","job_status":"start","queue_duration_s":1.732858,"scheduling_latency_s":1.732875} vm-test-run-nixbot-gitlab> gitlab # [ 283.592058] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.618Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f157fef9d471ae9c4db0ae77","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f157fef9d471ae9c4db0ae77: done: 0.014316 sec","job_status":"done","queue_duration_s":1.732858,"scheduling_latency_s":1.732875,"redis_calls":13,"redis_duration_s":0.001504,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014316,"completed_at":"2026-10-04T11:31:03.618Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 283.632604] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.619Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fbf89a7abf45ec0b5bfdb3ec","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fbf89a7abf45ec0b5bfdb3ec: start","job_status":"start","queue_duration_s":1.74834,"scheduling_latency_s":1.748357} vm-test-run-nixbot-gitlab> gitlab # [ 283.645648] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fbf89a7abf45ec0b5bfdb3ec","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fbf89a7abf45ec0b5bfdb3ec: done: 0.01431 sec","job_status":"done","queue_duration_s":1.74834,"scheduling_latency_s":1.748357,"redis_calls":13,"redis_duration_s":0.001497,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01431,"completed_at":"2026-10-04T11:31:03.634Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 283.686589] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2c4a73d7c2c3cd050a3c41a","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2c4a73d7c2c3cd050a3c41a: start","job_status":"start","queue_duration_s":1.763801,"scheduling_latency_s":1.76382} vm-test-run-nixbot-gitlab> gitlab # [ 283.699344] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2c4a73d7c2c3cd050a3c41a","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2c4a73d7c2c3cd050a3c41a: done: 0.014409 sec","job_status":"done","queue_duration_s":1.763801,"scheduling_latency_s":1.76382,"redis_calls":13,"redis_duration_s":0.001499,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000973,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014409,"completed_at":"2026-10-04T11:31:03.649Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 283.740401] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.651Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cc21d5b7adca5a778552015","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cc21d5b7adca5a778552015: start","job_status":"start","queue_duration_s":1.779614,"scheduling_latency_s":1.779632} vm-test-run-nixbot-gitlab> gitlab # [ 283.753168] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.665Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cc21d5b7adca5a778552015","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cc21d5b7adca5a778552015: done: 0.014417 sec","job_status":"done","queue_duration_s":1.779614,"scheduling_latency_s":1.779632,"redis_calls":13,"redis_duration_s":0.001598,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000555,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001043,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014417,"completed_at":"2026-10-04T11:31:03.665Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 283.794772] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.666Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f25e81167b14e95f32eae45d","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f25e81167b14e95f32eae45d: start","job_status":"start","queue_duration_s":1.795099,"scheduling_latency_s":1.795116} vm-test-run-nixbot-gitlab> gitlab # [ 283.807746] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.681Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f25e81167b14e95f32eae45d","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f25e81167b14e95f32eae45d: done: 0.014467 sec","job_status":"done","queue_duration_s":1.795099,"scheduling_latency_s":1.795116,"redis_calls":13,"redis_duration_s":0.001549,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014467,"completed_at":"2026-10-04T11:31:03.681Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 283.848794] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.682Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"810efdcc2ae88825ce0934c5","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-810efdcc2ae88825ce0934c5: start","job_status":"start","queue_duration_s":1.810677,"scheduling_latency_s":1.810693} vm-test-run-nixbot-gitlab> gitlab # [ 283.861631] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.696Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"810efdcc2ae88825ce0934c5","created_at":"2026-10-04T11:31:01.579Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-3a3a3ffad36e449f","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b9f8d27319b365bb8894b6a4d4f24ad","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.871Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-810efdcc2ae88825ce0934c5: done: 0.014272 sec","job_status":"done","queue_duration_s":1.810677,"scheduling_latency_s":1.810693,"redis_calls":13,"redis_duration_s":0.001473,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014272,"completed_at":"2026-10-04T11:31:03.696Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 283.902253] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.697Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"4c1c6ec4d575aeff9e9d1529","created_at":"2026-10-04T11:31:01.870Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3cb8456c7b5151302f09c0c9007d13a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.881Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-4c1c6ec4d575aeff9e9d1529: start","job_status":"start","queue_duration_s":1.816236,"scheduling_latency_s":1.816253} vm-test-run-nixbot-gitlab> gitlab # [ 283.917408] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.705Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 283.921240] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.710Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"4c1c6ec4d575aeff9e9d1529","created_at":"2026-10-04T11:31:01.870Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-d6a30651d14542f2","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3cb8456c7b5151302f09c0c9007d13a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:01.881Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-4c1c6ec4d575aeff9e9d1529: done: 0.012783 sec","job_status":"done","queue_duration_s":1.816236,"scheduling_latency_s":1.816253,"redis_calls":7,"redis_duration_s":0.000954,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000685,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000269,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007798,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012783,"completed_at":"2026-10-04T11:31:03.710Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 283.963213] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.712Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"33c90e95e04c3074da3f2611","created_at":"2026-10-04T11:31:02.099Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a945009e43314fb0","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b05145b3c3d870ecf5299b943b0f53f4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:02.109Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-33c90e95e04c3074da3f2611: start","job_status":"start","queue_duration_s":1.602019,"scheduling_latency_s":1.602037} vm-test-run-nixbot-gitlab> gitlab # [ 283.977150] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.723Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 283.978850] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.732Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"33c90e95e04c3074da3f2611","created_at":"2026-10-04T11:31:02.099Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a945009e43314fb0","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b05145b3c3d870ecf5299b943b0f53f4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:02.109Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-33c90e95e04c3074da3f2611: done: 0.020677 sec","job_status":"done","queue_duration_s":1.602019,"scheduling_latency_s":1.602037,"redis_calls":10,"redis_duration_s":0.001413,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000585,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000828,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020677,"completed_at":"2026-10-04T11:31:03.732Z","load_balancing_strategy":"primary","db_duration_s":0.002749,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 284.018821] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.733Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1955778f0f637cdcb664961b","created_at":"2026-10-04T11:31:02.099Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a945009e43314fb0","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b05145b3c3d870ecf5299b943b0f53f4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:02.109Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1955778f0f637cdcb664961b: start","job_status":"start","queue_duration_s":1.623839,"scheduling_latency_s":1.623856} vm-test-run-nixbot-gitlab> gitlab # [ 284.031760] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.744Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 284.033621] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.752Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1955778f0f637cdcb664961b","created_at":"2026-10-04T11:31:02.099Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a945009e43314fb0","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b05145b3c3d870ecf5299b943b0f53f4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:02.109Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1955778f0f637cdcb664961b: done: 0.018645 sec","job_status":"done","queue_duration_s":1.623839,"scheduling_latency_s":1.623856,"redis_calls":10,"redis_duration_s":0.001387,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000612,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000775,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013045,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018645,"completed_at":"2026-10-04T11:31:03.752Z","load_balancing_strategy":"primary","db_duration_s":0.001718,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 284.073116] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.753Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4efc96d523ffa7e951e6591d","created_at":"2026-10-04T11:31:02.099Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a945009e43314fb0","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b05145b3c3d870ecf5299b943b0f53f4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:02.109Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4efc96d523ffa7e951e6591d: start","job_status":"start","queue_duration_s":1.643667,"scheduling_latency_s":1.643684} vm-test-run-nixbot-gitlab> gitlab # [ 284.085851] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.763Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 284.087540] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.772Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4efc96d523ffa7e951e6591d","created_at":"2026-10-04T11:31:02.099Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a945009e43314fb0","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b05145b3c3d870ecf5299b943b0f53f4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:02.109Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4efc96d523ffa7e951e6591d: done: 0.018576 sec","job_status":"done","queue_duration_s":1.643667,"scheduling_latency_s":1.643684,"redis_calls":10,"redis_duration_s":0.001372,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000595,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000777,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01294,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018576,"completed_at":"2026-10-04T11:31:03.772Z","load_balancing_strategy":"primary","db_duration_s":0.001632,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 284.127433] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.773Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9369c5cb7c2cf37575804a4b","created_at":"2026-10-04T11:31:02.099Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a945009e43314fb0","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b05145b3c3d870ecf5299b943b0f53f4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:02.109Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9369c5cb7c2cf37575804a4b: start","job_status":"start","queue_duration_s":1.663363,"scheduling_latency_s":1.66338} vm-test-run-nixbot-gitlab> gitlab # [ 284.140255] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.783Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 284.141983] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:03.791Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9369c5cb7c2cf37575804a4b","created_at":"2026-10-04T11:31:02.099Z","trace_propagation_headers":{"sentry-trace":"7e8ef66a3e6641ccb69ca7618f65f326-a945009e43314fb0","baggage":"sentry-trace_id=7e8ef66a3e6641ccb69ca7618f65f326,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b05145b3c3d870ecf5299b943b0f53f4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:02.109Z","job_size_bytes":2,"pid":1899,"sidekiq_tid":"mal3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9369c5cb7c2cf37575804a4b: done: 0.018257 sec","job_status":"done","queue_duration_s":1.663363,"scheduling_latency_s":1.66338,"redis_calls":10,"redis_duration_s":0.00142,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000625,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000795,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012917,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018257,"completed_at":"2026-10-04T11:31:03.791Z","load_balancing_strategy":"primary","db_duration_s":0.001556,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 292.577274] postgres[3179]: [3179] ERROR: relation "table_without_models" does not exist vm-test-run-nixbot-gitlab> gitlab # [ 292.579212] postgres[3179]: [3179] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') vm-test-run-nixbot-gitlab> gitlab # [ 298.177628] runuser[2833]: pam_unix(runuser:session): session closed for user gitlab vm-test-run-nixbot-gitlab> gitlab # [ 298.178702] setup-gitlab-start[2803]: + systemctl restart gitlab-sidekiq.service vm-test-run-nixbot-gitlab> gitlab # [ 298.203740] systemd[1]: Stopping gitlab-sidekiq.service... vm-test-run-nixbot-gitlab> gitlab # [ 298.209495] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:20.107Z","message":"Shutting down"} vm-test-run-nixbot-gitlab> gitlab # [ 298.211718] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:20.108Z","message":"Scheduler exiting..."} vm-test-run-nixbot-gitlab> gitlab # [ 298.214108] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:20.108Z","message":"Terminating quiet threads for default capsule"} vm-test-run-nixbot-gitlab> gitlab # [ 298.216834] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:20.109Z","message":"Scheduler exiting..."} vm-test-run-nixbot-gitlab> gitlab # [ 298.711237] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:20.610Z","message":"Pausing to allow jobs to finish..."} vm-test-run-nixbot-gitlab> gitlab # [ 302.214529] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:24.112Z","message":"Bye!"} vm-test-run-nixbot-gitlab> gitlab # [ 302.239740] sidekiq-cluster[1899]: {"severity":"INFO","time":"2026-10-04T11:31:24.138Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1899,"worker_id":"sidekiq_0","memwd_rss_bytes":951341056,"retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 302.962973] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 302.964731] systemd[1]: Stopped gitlab-sidekiq.service. vm-test-run-nixbot-gitlab> gitlab # [ 302.965964] systemd[1]: gitlab-sidekiq.service: Consumed 1min 50.664s CPU time over 3min 18.298s wall clock time, 940.9M memory peak, 4.5M read from disk. vm-test-run-nixbot-gitlab> gitlab # [ 302.969199] systemd[1]: Started gitlab-sidekiq.service. vm-test-run-nixbot-gitlab> gitlab # [ 302.971326] setup-gitlab-start[2803]: + mkdir -p /var/lib/secrets vm-test-run-nixbot-gitlab> gitlab # [ 302.974237] setup-gitlab-start[2803]: + echo nixbot-test-token-of-20-chars vm-test-run-nixbot-gitlab> gitlab # [ 302.975846] setup-gitlab-start[2803]: + chmod 644 /var/lib/secrets/gitlab-token vm-test-run-nixbot-gitlab> gitlab # [ 302.977910] setup-gitlab-start[2803]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars vm-test-run-nixbot-gitlab> gitlab # [ 303.316072] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:25 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 313 vm-test-run-nixbot-gitlab> gitlab # [ 303.994827] gitaly[910]: time="2026-10-04T11:31:25.893Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M43AXPXG69BJ0ZNHZBXN0NRM error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T11:31:35.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:25.892" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 303.995172] gitaly[910]: time="2026-10-04T11:31:25.893Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M43AXPXG69BJ0ZNHZBXN0NRM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T11:31:35.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:25.892" grpc.time_ms=0.532 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 304.123948] gitaly[910]: time="2026-10-04T11:31:26.022Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M43AXPXG69BJ0ZNHZBXN0NRM error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T11:31:36.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:26.022" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 304.141292] gitaly[910]: time="2026-10-04T11:31:26.023Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M43AXPXG69BJ0ZNHZBXN0NRM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T11:31:36.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:26.022" grpc.time_ms=0.408 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 304.165889] gitaly[910]: time="2026-10-04T11:31:26.064Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M43AXPXG69BJ0ZNHZBXN0NRM diskcache=e68b2bd1-abe8-426b-925c-a62e20856454 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-04T11:31:36.011" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:26.031" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 304.182920] gitaly[910]: time="2026-10-04T11:31:26.065Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1442 command.oublock=32 command.real_time_ms=20 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M43AXPXG69BJ0ZNHZBXN0NRM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-04T11:31:36.011" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:26.031" grpc.time_ms=33.332 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 304.204418] gitaly[910]: time="2026-10-04T11:31:26.076Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M43AXPXG69BJ0ZNHZBXN0NRM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T11:31:36.055" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:26.075" pid=910 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 304.221164] gitaly[910]: time="2026-10-04T11:31:26.076Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M43AXPXG69BJ0ZNHZBXN0NRM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T11:31:36.055" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:26.075" grpc.time_ms=1.005 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 304.235997] gitaly[910]: time="2026-10-04T11:31:26.088Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AXPXG69BJ0ZNHZBXN0NRM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T11:31:36.060" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:26.080" grpc.time_ms=7.188 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 304.254876] gitaly[910]: time="2026-10-04T11:31:26.153Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M43AXPXG69BJ0ZNHZBXN0NRM error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T11:31:36.132" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:26.153" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 304.267572] gitaly[910]: time="2026-10-04T11:31:26.154Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M43AXPXG69BJ0ZNHZBXN0NRM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T11:31:36.132" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:26.153" grpc.time_ms=0.393 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 304.284756] gitaly[910]: time="2026-10-04T11:31:26.183Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M43AXPXG69BJ0ZNHZBXN0NRM diskcache=7b8875d2-bfdb-4643-a0dd-42c98630c02e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-04T11:31:36.135" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:26.156" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 304.296965] gitaly[910]: time="2026-10-04T11:31:26.184Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1440 command.oublock=32 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M43AXPXG69BJ0ZNHZBXN0NRM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-04T11:31:36.135" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:26.156" grpc.time_ms=27.371 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 304.311747] gitaly[910]: time="2026-10-04T11:31:26.191Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M43AXPXG69BJ0ZNHZBXN0NRM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T11:31:36.169" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:26.190" pid=910 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 304.323087] gitaly[910]: time="2026-10-04T11:31:26.191Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M43AXPXG69BJ0ZNHZBXN0NRM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T11:31:36.169" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:26.190" grpc.time_ms=0.876 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 304.334728] gitaly[910]: time="2026-10-04T11:31:26.201Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M43AXPXG69BJ0ZNHZBXN0NRM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T11:31:36.174" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:26.194" grpc.time_ms=7 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 304.349361] gitaly[910]: time="2026-10-04T11:31:26.231Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M43AXPXG69BJ0ZNHZBXN0NRM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T11:31:36.210" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:26.230" grpc.time_ms=0.142 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 304.360617] gitaly[910]: time="2026-10-04T11:31:26.238Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M43AXPXG69BJ0ZNHZBXN0NRM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-04T11:31:36.217" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:26.237" grpc.time_ms=0.308 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 304.928796] gitaly[910]: time="2026-10-04T11:31:26.827Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AXPXG69BJ0ZNHZBXN0NRM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T11:31:36.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:26.817" grpc.time_ms=10.061 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 305.573184] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:27 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 2238 vm-test-run-nixbot-gitlab> gitlab # [ 305.576500] setup-gitlab-start[3250]: {"id":1,"message":null,"error":null} vm-test-run-nixbot-gitlab> gitlab # [ 308.069188] sidekiq-cluster[3238]: {"severity":"INFO","time":"2026-10-04T11:31:29.939Z","message":"Starting cluster with 1 processes"} vm-test-run-nixbot-gitlab> gitlab # [ 309.141247] redis[790]: 10 changes in 300 seconds. Saving... vm-test-run-nixbot-gitlab> gitlab # [ 309.141856] redis[790]: Background saving started by pid 3313 vm-test-run-nixbot-gitlab> gitlab # [ 309.153665] redis[3313]: BGSAVE done, 447 keys saved, 0 keys skipped, 113203 bytes written. vm-test-run-nixbot-gitlab> gitlab # [ 309.157374] redis[3313]: DB saved on disk vm-test-run-nixbot-gitlab> gitlab # [ 309.158615] redis[3313]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB vm-test-run-nixbot-gitlab> gitlab # [ 309.242018] redis[790]: Background saving terminated with success vm-test-run-nixbot-gitlab> gitlab # [ 310.721564] gitaly[910]: time="2026-10-04T11:31:32.619Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AXY0F34YKEP71C4G81NR2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T11:31:42.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:32.609" grpc.time_ms=10.214 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 310.800602] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:32 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 203 vm-test-run-nixbot-gitlab> gitlab # [ 310.804361] setup-gitlab-start[2803]: + mkdir -p /tmp/test-flake vm-test-run-nixbot-gitlab> gitlab # [ 310.806905] setup-gitlab-start[2803]: + cd /tmp/test-flake vm-test-run-nixbot-gitlab> gitlab # [ 310.808301] setup-gitlab-start[2803]: + git init -b master vm-test-run-nixbot-gitlab> gitlab # [ 310.822316] setup-gitlab-start[3325]: Initialized empty Git repository in /tmp/test-flake/.git/ vm-test-run-nixbot-gitlab> gitlab # [ 310.824233] setup-gitlab-start[2803]: + git config user.name test vm-test-run-nixbot-gitlab> gitlab # [ 310.826213] setup-gitlab-start[2803]: + git config user.email test@example.com vm-test-run-nixbot-gitlab> gitlab # [ 310.829744] setup-gitlab-start[2803]: + cat vm-test-run-nixbot-gitlab> gitlab # [ 310.832728] setup-gitlab-start[2803]: + git add flake.nix vm-test-run-nixbot-gitlab> gitlab # [ 310.836862] setup-gitlab-start[2803]: + git commit -m 'initial commit' vm-test-run-nixbot-gitlab> gitlab # [ 310.851838] setup-gitlab-start[3330]: [master (root-commit) baa7927] initial commit vm-test-run-nixbot-gitlab> gitlab # [ 310.853589] setup-gitlab-start[3330]: 1 file changed, 10 insertions(+) vm-test-run-nixbot-gitlab> gitlab # [ 310.854987] setup-gitlab-start[3330]: create mode 100644 flake.nix vm-test-run-nixbot-gitlab> gitlab # [ 310.856453] setup-gitlab-start[2803]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git vm-test-run-nixbot-gitlab> gitlab # [ 310.858915] setup-gitlab-start[2803]: + git push -u origin master vm-test-run-nixbot-gitlab> gitlab # [ 310.927850] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:32 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 44 vm-test-run-nixbot-gitlab> gitlab # [ 310.949373] postgres[858]: [858] LOG: checkpoint starting: time vm-test-run-nixbot-gitlab> gitlab # [ 311.550106] gitaly[910]: time="2026-10-04T11:31:33.449Z" level=info msg="burden monitor stats" pid=910 vm-test-run-nixbot-gitlab> gitlab # [ 311.955519] gitaly[910]: time="2026-10-04T11:31:33.853Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43AXYAV1PBCF1G9AGV6ZF13 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:33.841" grpc.time_ms=12.308 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 311.979544] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:33 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 1027 vm-test-run-nixbot-gitlab> gitlab # [ 312.284299] gitaly[910]: time="2026-10-04T11:31:34.183Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T11:31:44.003" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:34.173" grpc.time_ms=9.202 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 312.353705] gitaly[910]: time="2026-10-04T11:31:34.252Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-04T11:32:04.041" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-04T11:31:34.241" grpc.time_ms=10.714 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 312.455889] gitaly[910]: time="2026-10-04T11:31:34.354Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 duration_ms=344ns method=POST pid=910 status=200 url="http://unix/api/v4/internal/allowed" vm-test-run-nixbot-gitlab> gitlab # [ 312.494717] gitaly[910]: time="2026-10-04T11:31:34.393Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 duration_ms=38ns method=POST pid=910 status=200 url="http://unix/api/v4/internal/pre_receive" vm-test-run-nixbot-gitlab> gitlab # [ 312.500336] gitaly[910]: time="2026-10-04T11:31:34.394Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:31:34.009" grpc.time_ms=384.731 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 312.532636] gitaly[910]: time="2026-10-04T11:31:34.431Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:31:34.429" grpc.time_ms=1.591 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 312.571480] gitaly[910]: time="2026-10-04T11:31:34.470Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:31:34.469" grpc.time_ms=0.641 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 312.618286] gitaly[910]: time="2026-10-04T11:31:34.517Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:31:34.516" grpc.time_ms=0.782 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 312.807575] gitaly[910]: time="2026-10-04T11:31:34.706Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T11:31:44.005" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:34.696" grpc.time_ms=10.268 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 312.836911] gitaly[910]: time="2026-10-04T11:31:34.735Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1435 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-04T11:31:44.024" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:31:34.714" grpc.time_ms=20.796 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 312.862268] gitaly[910]: time="2026-10-04T11:31:34.747Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 duration_ms=181ns method=POST pid=910 status=200 url="http://unix/api/v4/internal/post_receive" vm-test-run-nixbot-gitlab> gitlab # [ 312.867153] gitaly[910]: time="2026-10-04T11:31:34.748Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:31:34.565" grpc.time_ms=182.708 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 312.883529] gitaly[910]: time="2026-10-04T11:31:34.765Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 diskcache=a667dac0-efc7-4649-a751-cb7cdd7e7582 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:33.946" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 312.898068] systemd[1]: Finished setup-gitlab.service. vm-test-run-nixbot-gitlab> gitlab # [ 312.899083] gitlab-workhorse[817]: time="2026-10-04T11:31:34Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 vm-test-run-nixbot-gitlab> gitlab # [ 312.901518] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:34 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 898 vm-test-run-nixbot-gitlab> gitlab # [ 312.903954] setup-gitlab-start[3334]: To http://gitlab/root/test-flake.git vm-test-run-nixbot-gitlab> gitlab # [ 312.905219] setup-gitlab-start[3334]: * [new branch] master -> master vm-test-run-nixbot-gitlab> gitlab # [ 312.906301] setup-gitlab-start[3334]: branch 'master' set up to track 'origin/master'. vm-test-run-nixbot-gitlab> gitlab # [ 312.907525] gitaly[910]: time="2026-10-04T11:31:34.766Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=221 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13697 command.oublock=32 command.real_time_ms=813 command.spawn_token_fork_ms=0 command.system_time_ms=132 command.user_time_ms=89 component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:33.946" grpc.time_ms=819.016 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 312.922858] systemd[1]: Starting nixbot CI service... vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit setup-gitlab.service, in 313.84 seconds) vm-test-run-nixbot-gitlab> gitlab: waiting for unit nixbot.service vm-test-run-nixbot-gitlab> gitlab # [ 314.028190] nixbot[3426]: {"ts": "2026-10-04T11:31:35.926Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} vm-test-run-nixbot-gitlab> gitlab # [ 314.061478] nixbot[3426]: {"ts": "2026-10-04T11:31:35.960Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} vm-test-run-nixbot-gitlab> gitlab # [ 314.084113] nixbot[3426]: {"ts": "2026-10-04T11:31:35.983Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} vm-test-run-nixbot-gitlab> gitlab # [ 314.088540] nixbot[3426]: {"ts": "2026-10-04T11:31:35.987Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} vm-test-run-nixbot-gitlab> gitlab # [ 314.092566] nixbot[3426]: {"ts": "2026-10-04T11:31:35.991Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} vm-test-run-nixbot-gitlab> gitlab # [ 314.098920] nixbot[3426]: {"ts": "2026-10-04T11:31:35.997Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} vm-test-run-nixbot-gitlab> gitlab # [ 314.103393] nixbot[3426]: {"ts": "2026-10-04T11:31:36.002Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} vm-test-run-nixbot-gitlab> gitlab # [ 314.107176] nixbot[3426]: {"ts": "2026-10-04T11:31:36.004Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} vm-test-run-nixbot-gitlab> gitlab # [ 314.110914] nixbot[3426]: {"ts": "2026-10-04T11:31:36.009Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} vm-test-run-nixbot-gitlab> gitlab # [ 314.114567] nixbot[3426]: {"ts": "2026-10-04T11:31:36.011Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} vm-test-run-nixbot-gitlab> gitlab # [ 314.118224] nixbot[3426]: {"ts": "2026-10-04T11:31:36.016Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} vm-test-run-nixbot-gitlab> gitlab # [ 314.124079] nixbot[3426]: {"ts": "2026-10-04T11:31:36.022Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} vm-test-run-nixbot-gitlab> gitlab # [ 314.127570] nixbot[3426]: {"ts": "2026-10-04T11:31:36.025Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} vm-test-run-nixbot-gitlab> gitlab # [ 314.131333] nixbot[3426]: {"ts": "2026-10-04T11:31:36.027Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} vm-test-run-nixbot-gitlab> gitlab # [ 314.135152] nixbot[3426]: {"ts": "2026-10-04T11:31:36.031Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} vm-test-run-nixbot-gitlab> gitlab # [ 314.138818] nixbot[3426]: {"ts": "2026-10-04T11:31:36.032Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} vm-test-run-nixbot-gitlab> gitlab # [ 314.142630] nixbot[3426]: {"ts": "2026-10-04T11:31:36.033Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} vm-test-run-nixbot-gitlab> gitlab # [ 314.146327] nixbot[3426]: {"ts": "2026-10-04T11:31:36.034Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} vm-test-run-nixbot-gitlab> gitlab # [ 314.149749] nixbot[3426]: {"ts": "2026-10-04T11:31:36.039Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} vm-test-run-nixbot-gitlab> gitlab # [ 314.153267] nixbot[3426]: {"ts": "2026-10-04T11:31:36.040Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} vm-test-run-nixbot-gitlab> gitlab # [ 314.156819] nixbot[3426]: {"ts": "2026-10-04T11:31:36.042Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} vm-test-run-nixbot-gitlab> gitlab # [ 314.160248] nixbot[3426]: {"ts": "2026-10-04T11:31:36.047Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} vm-test-run-nixbot-gitlab> gitlab # [ 314.163543] nixbot[3426]: {"ts": "2026-10-04T11:31:36.049Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} vm-test-run-nixbot-gitlab> gitlab # [ 314.166830] nixbot[3426]: {"ts": "2026-10-04T11:31:36.050Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} vm-test-run-nixbot-gitlab> gitlab # [ 314.169833] nixbot[3426]: {"ts": "2026-10-04T11:31:36.052Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} vm-test-run-nixbot-gitlab> gitlab # [ 314.172796] nixbot[3426]: {"ts": "2026-10-04T11:31:36.055Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} vm-test-run-nixbot-gitlab> gitlab # [ 314.175672] nixbot[3426]: {"ts": "2026-10-04T11:31:36.056Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} vm-test-run-nixbot-gitlab> gitlab # [ 314.178726] nixbot[3426]: {"ts": "2026-10-04T11:31:36.058Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"} vm-test-run-nixbot-gitlab> gitlab # [ 314.181735] nixbot[3426]: {"ts": "2026-10-04T11:31:36.059Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"} vm-test-run-nixbot-gitlab> gitlab # [ 314.184653] nixbot[3426]: {"ts": "2026-10-04T11:31:36.073Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"} vm-test-run-nixbot-gitlab> gitlab # [ 314.187482] nixbot[3426]: {"ts": "2026-10-04T11:31:36.078Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"} vm-test-run-nixbot-gitlab> gitlab # [ 314.190266] nixbot[3426]: {"ts": "2026-10-04T11:31:36.080Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"} vm-test-run-nixbot-gitlab> gitlab # [ 314.193102] nixbot[3426]: {"ts": "2026-10-04T11:31:36.081Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"} vm-test-run-nixbot-gitlab> gitlab # [ 314.195738] nixbot[3426]: {"ts": "2026-10-04T11:31:36.085Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"} vm-test-run-nixbot-gitlab> gitlab # [ 314.198683] nixbot[3426]: {"ts": "2026-10-04T11:31:36.086Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"} vm-test-run-nixbot-gitlab> gitlab # [ 314.201419] nixbot[3426]: {"ts": "2026-10-04T11:31:36.092Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"} vm-test-run-nixbot-gitlab> gitlab # [ 314.204152] nixbot[3426]: {"ts": "2026-10-04T11:31:36.095Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"} vm-test-run-nixbot-gitlab> gitlab # [ 314.206667] nixbot[3426]: {"ts": "2026-10-04T11:31:36.099Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 37, "migration": "effect_skip_key"} vm-test-run-nixbot-gitlab> gitlab # [ 314.621630] nixbot[3426]: INFO: Started server process [3426] vm-test-run-nixbot-gitlab> gitlab # [ 314.623188] nixbot[3426]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) vm-test-run-nixbot-gitlab> gitlab # [ 314.644872] nixbot[3426]: INFO: ::ffff:127.0.0.1:50668 - "GET /health HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab # [ 314.652791] systemd[1]: Started nixbot CI service. vm-test-run-nixbot-gitlab> gitlab # [ 314.654108] systemd[1]: Reached target Multi-User System. vm-test-run-nixbot-gitlab> gitlab # [ 314.655322] systemd[1]: Startup finished in 693ms (kernel) + 4.599s (initrd) + 5min 9.358s (userspace) = 5min 14.651s. vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit nixbot.service, in 2.08 seconds) vm-test-run-nixbot-gitlab> gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health vm-test-run-nixbot-gitlab> gitlab # [ 315.518022] nixbot[3426]: INFO: ::ffff:127.0.0.1:50674 - "GET /health HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.03 seconds) vm-test-run-nixbot-gitlab> subtest: project discovered and webhook registered vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks vm-test-run-nixbot-gitlab> gitlab # [ 315.642515] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:37 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 92 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.13 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 315.668125] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:37 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 1020 vm-test-run-nixbot-gitlab> gitlab # [ 315.671979] nixbot[3426]: {"ts": "2026-10-04T11:31:37.565Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 315.698383] nixbot[3426]: {"ts": "2026-10-04T11:31:37.597Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} vm-test-run-nixbot-gitlab> gitlab # [ 315.772774] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:37 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 66 vm-test-run-nixbot-gitlab> gitlab # [ 315.776309] nixbot[3426]: {"ts": "2026-10-04T11:31:37.673Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 315.779974] nixbot[3426]: {"ts": "2026-10-04T11:31:37.675Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} vm-test-run-nixbot-gitlab> gitlab # [ 315.954822] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 175 vm-test-run-nixbot-gitlab> gitlab # [ 315.958462] nixbot[3426]: {"ts": "2026-10-04T11:31:37.855Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 316.242548] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:38 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 276 vm-test-run-nixbot-gitlab> gitlab # [ 316.246667] nixbot[3426]: {"ts": "2026-10-04T11:31:38.142Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 316.333537] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 79 vm-test-run-nixbot-gitlab> gitlab # [ 316.337266] nixbot[3426]: {"ts": "2026-10-04T11:31:38.233Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 316.410698] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:38 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 71 vm-test-run-nixbot-gitlab> gitlab # [ 316.414637] nixbot[3426]: {"ts": "2026-10-04T11:31:38.310Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 316.492574] gitaly[910]: time="2026-10-04T11:31:38.391Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AY3PG45EZQ6V0734S335M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-10-04T11:32:08.073" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:31:38.374" grpc.time_ms=16.136 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 316.524284] gitaly[910]: time="2026-10-04T11:31:38.423Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AY3PG45EZQ6V0734S335M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:32:08.121" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:31:38.422" grpc.time_ms=0.718 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 316.539294] gitaly[910]: time="2026-10-04T11:31:38.438Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M43AY3PG45EZQ6V0734S335M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-10-04T11:31:48.006" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:31:38.427" grpc.time_ms=10.745 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 316.574643] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 152 vm-test-run-nixbot-gitlab> gitlab # [ 316.577647] nixbot[3426]: {"ts": "2026-10-04T11:31:38.474Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks vm-test-run-nixbot-gitlab> gitlab # [ 316.673485] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 92 vm-test-run-nixbot-gitlab> gitlab # [ 316.676128] nixbot[3426]: {"ts": "2026-10-04T11:31:38.573Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 316.681434] nixbot[3426]: {"ts": "2026-10-04T11:31:38.579Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} vm-test-run-nixbot-gitlab> gitlab # [ 316.800295] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 120 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.16 seconds) vm-test-run-nixbot-gitlab> (finished: subtest: project discovered and webhook registered, in 1.28 seconds) vm-test-run-nixbot-gitlab> subtest: push delivers the webhook vm-test-run-nixbot-gitlab> gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails' vm-test-run-nixbot-gitlab> gitlab # [ 316.831872] gitaly[910]: time="2026-10-04T11:31:38.730Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43AY416TMR8ZE7PEJPYVBYZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:38.719" grpc.time_ms=11.64 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 316.851569] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 69 vm-test-run-nixbot-gitlab> gitlab # [ 316.959674] gitaly[910]: time="2026-10-04T11:31:38.858Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M43AY43C8RZ3C2H57SFFSNFC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:38.847" pid=910 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 316.976648] gitaly[910]: time="2026-10-04T11:31:38.858Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M43AY43C8RZ3C2H57SFFSNFC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:38.847" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 316.998471] gitlab-workhorse[817]: time="2026-10-04T11:31:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 vm-test-run-nixbot-gitlab> gitlab # [ 317.001317] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 127 vm-test-run-nixbot-gitlab> gitlab # [ 317.005125] gitaly[910]: time="2026-10-04T11:31:38.859Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M43AY43C8RZ3C2H57SFFSNFC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:38.847" grpc.time_ms=11.452 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 317.104103] gitaly[910]: time="2026-10-04T11:31:39.002Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M43AY47GQ2ZYRZJ2N9X6FJ5C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:38.982" pid=910 remote_ip="2001:db8:1::1" request_sha=e4a39bd46c851836fe1f5338af1b3a93b6b2aee3 response_bytes=223 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 317.121367] gitaly[910]: time="2026-10-04T11:31:39.002Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1135 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=13 component=gitaly.UnaryServerInterceptor correlation_id=01M43AY47GQ2ZYRZJ2N9X6FJ5C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:38.982" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 317.144568] gitlab-workhorse[817]: time="2026-10-04T11:31:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=223 vm-test-run-nixbot-gitlab> gitlab # [ 317.147628] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 155 vm-test-run-nixbot-gitlab> gitlab # [ 317.150881] gitaly[910]: time="2026-10-04T11:31:39.003Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1135 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=13 component=gitaly.UnaryServerInterceptor correlation_id=01M43AY47GQ2ZYRZJ2N9X6FJ5C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:38.982" grpc.time_ms=20.198 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 317.226482] gitaly[910]: time="2026-10-04T11:31:39.124Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M43AY4DJ6FGCB7PV33RPFCCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:39.114" grpc.time_ms=10.051 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 317.248168] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 66 vm-test-run-nixbot-gitlab> gitlab # [ 317.298302] gitaly[910]: time="2026-10-04T11:31:39.197Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M43AY4FPXD9ZCMSW5D4TSGMF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:39.186" pid=910 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 317.316301] gitaly[910]: time="2026-10-04T11:31:39.197Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AY4FPXD9ZCMSW5D4TSGMF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:39.186" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 317.339995] gitlab-workhorse[817]: time="2026-10-04T11:31:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 317.342972] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 71 vm-test-run-nixbot-gitlab> gitlab # [ 317.345957] gitaly[910]: time="2026-10-04T11:31:39.197Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AY4FPXD9ZCMSW5D4TSGMF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:39.186" grpc.time_ms=11.002 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 317.442601] gitaly[910]: time="2026-10-04T11:31:39.341Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43AY4M538K64M942XYJ8P89 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:39.331" grpc.time_ms=10.212 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 317.464289] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 72 vm-test-run-nixbot-gitlab> gitlab # [ 317.532469] gitaly[910]: time="2026-10-04T11:31:39.430Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M43AY4PM07V4PE8WPBG4ZC75 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:39.411" pid=910 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 317.549634] gitaly[910]: time="2026-10-04T11:31:39.430Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1101 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M43AY4PM07V4PE8WPBG4ZC75 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:39.411" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 317.572181] gitlab-workhorse[817]: time="2026-10-04T11:31:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 vm-test-run-nixbot-gitlab> gitlab # [ 317.575478] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 82 vm-test-run-nixbot-gitlab> gitlab # [ 317.578784] gitaly[910]: time="2026-10-04T11:31:39.431Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1101 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M43AY4PM07V4PE8WPBG4ZC75 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:39.411" grpc.time_ms=19.639 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 317.888974] gitaly[910]: time="2026-10-04T11:31:39.787Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AY4VG20322ADMX7R8413B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:32:09.583" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:31:39.783" grpc.time_ms=3.863 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 317.964417] gitaly[910]: time="2026-10-04T11:31:39.863Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43AY4VG20322ADMX7R8413B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-04T11:32:09.651" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:31:39.852" grpc.time_ms=10.961 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 318.017670] gitaly[910]: time="2026-10-04T11:31:39.916Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43AY4VG20322ADMX7R8413B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T11:31:49.216" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:31:39.907" grpc.time_ms=9.386 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 318.040861] gitaly[910]: time="2026-10-04T11:31:39.939Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M43AY4VG20322ADMX7R8413B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T11:31:49.240" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:31:39.930" grpc.time_ms=9.293 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 318.161330] gitaly[910]: time="2026-10-04T11:31:40.060Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M43AY4VG20322ADMX7R8413B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-04T11:31:50.008" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:40.059" grpc.time_ms=0.343 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 318.182083] gitaly[910]: time="2026-10-04T11:31:40.081Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=569 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M43AY4VG20322ADMX7R8413B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-04T11:31:50.020" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:31:40.071" grpc.time_ms=9.309 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 318.234318] gitaly[910]: time="2026-10-04T11:31:40.133Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43AY4VG20322ADMX7R8413B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:31:40.132" grpc.time_ms=0.449 pid=910 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 318.281281] gitaly[910]: time="2026-10-04T11:31:40.180Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43AY4VG20322ADMX7R8413B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:31:40.179" grpc.time_ms=0.479 pid=910 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 318.303240] gitaly[910]: time="2026-10-04T11:31:40.202Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M43AY4VG20322ADMX7R8413B diskcache=71e9efdd-9f26-421c-b43b-239785e7af6f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T11:31:50.033" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:40.083" pid=910 ref=refs/keep-around/baa7927c76a37407367ce093ce7f4d35363a479f remote_ip="2001:db8:1::1" revision=baa7927c76a37407367ce093ce7f4d35363a479f span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 318.321630] gitaly[910]: time="2026-10-04T11:31:40.202Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=80 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5150 command.oublock=8 command.real_time_ms=113 command.spawn_token_fork_ms=0 command.system_time_ms=46 command.user_time_ms=34 component=gitaly.UnaryServerInterceptor correlation_id=01M43AY4VG20322ADMX7R8413B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T11:31:50.033" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:31:40.083" grpc.time_ms=118.442 pid=910 ref=refs/keep-around/baa7927c76a37407367ce093ce7f4d35363a479f remote_ip="2001:db8:1::1" revision=baa7927c76a37407367ce093ce7f4d35363a479f span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 318.341598] gitaly[910]: time="2026-10-04T11:31:40.206Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AY4VG20322ADMX7R8413B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:32:10.005" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:31:40.206" grpc.time_ms=0.403 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 318.523174] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/baa7927c76a37407367ce093ce7f4d35363a479f HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 916 vm-test-run-nixbot-gitlab> gitlab # [ 318.531627] nixbot[3426]: {"ts": "2026-10-04T11:31:40.427Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/baa7927c76a37407367ce093ce7f4d35363a479f \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 318.538306] nixbot[3426]: {"ts": "2026-10-04T11:31:40.437Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-baa7927c76a3-0"} vm-test-run-nixbot-gitlab> gitlab # [ 318.733461] nixbot[3426]: {"ts": "2026-10-04T11:31:40.632Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-baa7927c76a3-0", "argv0": "bwrap"} vm-test-run-nixbot-gitlab> gitlab # [ 318.863219] systemd[1]: Started Nix Daemon. vm-test-run-nixbot-gitlab> gitlab # [ 318.937226] nix-daemon[3586]: accepted connection from pid 3581, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 323.492420] nix-daemon[3586]: accepted connection from pid 3636, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 323.510566] nix-daemon[3586]: accepted connection from pid 3581, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 328.344657] nix-daemon[3586]: accepted connection from pid 3581, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 328.479413] gitaly[910]: time="2026-10-04T11:31:50.377Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AYFCS18SMKHTSYTYGZ006 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:32:20.176" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:31:50.376" grpc.time_ms=0.785 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 328.525620] gitaly[910]: time="2026-10-04T11:31:50.424Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43AYFCS18SMKHTSYTYGZ006 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T11:32:00.244" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:31:50.415" grpc.time_ms=9.352 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 328.618877] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:50 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/baa7927c76a37407367ce093ce7f4d35363a479f HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 219 vm-test-run-nixbot-gitlab> gitlab # [ 328.623756] nixbot[3426]: {"ts": "2026-10-04T11:31:50.518Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/baa7927c76a37407367ce093ce7f4d35363a479f \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 328.689026] gitaly[910]: time="2026-10-04T11:31:50.587Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AYFKV8Y1089KRVR97DWC0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:32:20.386" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:31:50.586" grpc.time_ms=0.851 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 328.727998] gitaly[910]: time="2026-10-04T11:31:50.626Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43AYFKV8Y1089KRVR97DWC0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T11:32:00.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:31:50.616" grpc.time_ms=10.447 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 328.802842] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:50 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/baa7927c76a37407367ce093ce7f4d35363a479f HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 178 vm-test-run-nixbot-gitlab> gitlab # [ 328.807346] nixbot[3426]: {"ts": "2026-10-04T11:31:50.703Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/baa7927c76a37407367ce093ce7f4d35363a479f \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 328.871259] gitaly[910]: time="2026-10-04T11:31:50.770Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AYFSTDNW3CZTNKZKM4Y3K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:32:20.568" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:31:50.769" grpc.time_ms=0.792 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 328.908677] gitaly[910]: time="2026-10-04T11:31:50.807Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M43AYFSTDNW3CZTNKZKM4Y3K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T11:32:00.106" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:31:50.796" grpc.time_ms=10.603 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 328.961114] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:50 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/baa7927c76a37407367ce093ce7f4d35363a479f HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 144 vm-test-run-nixbot-gitlab> gitlab # [ 328.964058] nixbot[3426]: {"ts": "2026-10-04T11:31:50.861Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/baa7927c76a37407367ce093ce7f4d35363a479f \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 329.386401] gitaly[910]: time="2026-10-04T11:31:51.285Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M43AYG97BZT3TTSBDR09Q2C1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:51.274" grpc.time_ms=10.545 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 329.408700] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:51 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 78 vm-test-run-nixbot-gitlab> gitlab # [ 329.460322] gitaly[910]: time="2026-10-04T11:31:51.359Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M43AYGBQZ0TTP9H1ZN3TNBM1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:51.349" pid=910 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 329.479644] gitaly[910]: time="2026-10-04T11:31:51.359Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AYGBQZ0TTP9H1ZN3TNBM1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:51.349" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 329.501706] gitlab-workhorse[817]: time="2026-10-04T11:31:51Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 329.504668] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:51 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 72 vm-test-run-nixbot-gitlab> gitlab # [ 329.507454] gitaly[910]: time="2026-10-04T11:31:51.359Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AYGBQZ0TTP9H1ZN3TNBM1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:51.349" grpc.time_ms=10.227 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 329.590292] gitaly[910]: time="2026-10-04T11:31:51.489Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M43AYGFSNZR9B2BTKGVCM6H5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:51.478" grpc.time_ms=10.158 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 329.613154] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:51 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 94 vm-test-run-nixbot-gitlab> gitlab # [ 329.689643] gitaly[910]: time="2026-10-04T11:31:51.588Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M43AYGJS3YGJ1QP27BXATTHD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:51.577" pid=910 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 329.707829] gitaly[910]: time="2026-10-04T11:31:51.588Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M43AYGJS3YGJ1QP27BXATTHD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:51.577" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 329.727479] gitlab-workhorse[817]: time="2026-10-04T11:31:51Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 329.729948] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:31:51 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 75 vm-test-run-nixbot-gitlab> gitlab # [ 329.732370] gitaly[910]: time="2026-10-04T11:31:51.589Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M43AYGJS3YGJ1QP27BXATTHD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:31:51.577" grpc.time_ms=11.575 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 349.644729] sidekiq-cluster[3304]: 2026-10-04T11:32:11.542Z pid=3304 tid=yf4 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3304, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>425316352} vm-test-run-nixbot-gitlab> gitlab # [ 349.919589] sidekiq-cluster[3304]: W, [2026-10-04T11:32:11.818536 #3304] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 350.063893] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:11.962Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} vm-test-run-nixbot-gitlab> gitlab # [ 353.074043] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:14.972Z","message":"GitLab reliable fetch activated!"} vm-test-run-nixbot-gitlab> gitlab # [ 353.076795] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:14.972Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} vm-test-run-nixbot-gitlab> gitlab # [ 387.402224] postgres[4174]: [4174] ERROR: relation "table_without_models" does not exist vm-test-run-nixbot-gitlab> gitlab # [ 387.402575] postgres[4174]: [4174] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') vm-test-run-nixbot-gitlab> gitlab # [ 392.172634] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:54.070Z","message":"Booted Rails 7.2.3.1 application in production environment"} vm-test-run-nixbot-gitlab> gitlab # [ 392.176947] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:54.075Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} vm-test-run-nixbot-gitlab> gitlab # [ 392.179056] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:54.076Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} vm-test-run-nixbot-gitlab> gitlab # [ 392.180942] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:54.076Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} vm-test-run-nixbot-gitlab> gitlab # [ 392.182933] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:54.077Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 75.61 seconds) vm-test-run-nixbot-gitlab> gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master vm-test-run-nixbot-gitlab> gitlab # [ 392.550809] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:32:54 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 82 vm-test-run-nixbot-gitlab> gitlab # [ 392.633759] gitaly[910]: time="2026-10-04T11:32:54.532Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M43B0E1JYPQZZT81YM410G6Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:32:54.521" grpc.time_ms=10.658 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # Enumerating objects: 5, done. vm-test-run-nixbot-gitlab> gitlab # [ 392.648067] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:32:54 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 82 vm-test-run-nixbot-gitlab> gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. vm-test-run-nixbot-gitlab> gitlab # Delta compression using up to 4 threads vm-test-run-nixbot-gitlab> gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. vm-test-run-nixbot-gitlab> gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 136.00 KiB/s, done. vm-test-run-nixbot-gitlab> gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) vm-test-run-nixbot-gitlab> gitlab # [ 392.869535] gitaly[910]: time="2026-10-04T11:32:54.768Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-04T11:33:24.558" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-04T11:32:54.758" grpc.time_ms=9.79 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 392.891963] gitaly[910]: time="2026-10-04T11:32:54.790Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M43B0E52BK28RD02GWGHJ25S duration_ms=97ns method=POST pid=910 status=200 url="http://unix/api/v4/internal/allowed" vm-test-run-nixbot-gitlab> gitlab # [ 392.975789] gitaly[910]: time="2026-10-04T11:32:54.874Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M43B0E52BK28RD02GWGHJ25S duration_ms=83ns method=POST pid=910 status=200 url="http://unix/api/v4/internal/pre_receive" vm-test-run-nixbot-gitlab> gitlab # [ 392.985519] gitaly[910]: time="2026-10-04T11:32:54.875Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:32:54.692" grpc.time_ms=182.39 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 393.012183] gitaly[910]: time="2026-10-04T11:32:54.911Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:32:54.910" grpc.time_ms=0.84 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 393.047422] gitaly[910]: time="2026-10-04T11:32:54.946Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:32:54.944" grpc.time_ms=2.093 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 393.096358] gitaly[910]: time="2026-10-04T11:32:54.995Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:32:54.994" grpc.time_ms=0.704 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 393.216534] gitaly[910]: time="2026-10-04T11:32:55.115Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M43B0E52BK28RD02GWGHJ25S duration_ms=70ns method=POST pid=910 status=200 url="http://unix/api/v4/internal/post_receive" vm-test-run-nixbot-gitlab> gitlab # [ 393.223099] gitaly[910]: time="2026-10-04T11:32:55.116Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:32:55.044" grpc.time_ms=71.591 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 393.241791] gitaly[910]: time="2026-10-04T11:32:55.140Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S diskcache=fbed2dd6-28c7-402d-882a-d40df13fcbe0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:32:54.631" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # To http://gitlab/root/test-flake.git vm-test-run-nixbot-gitlab> gitlab # [ 393.259701] gitaly[910]: time="2026-10-04T11:32:55.141Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=208 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=831488 command.minflt=13514 command.oublock=32 command.real_time_ms=504 command.spawn_token_fork_ms=0 command.system_time_ms=100 command.user_time_ms=108 component=gitaly.StreamServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:32:54.631" grpc.time_ms=509.524 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # baa7927..b1c994f [ 393.282075] gitlab-workhorse[817]: time="2026-10-04T11:32:55Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 vm-test-run-nixbot-gitlab> gitlab # master -> master[ 393.285318] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:32:55 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 578 vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.88 seconds) vm-test-run-nixbot-gitlab> gitlab: must succeed: git -C /tmp/test-flake rev-parse master vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55' vm-test-run-nixbot-gitlab> gitlab # [ 393.396062] nixbot[3426]: INFO: ::ffff:127.0.0.1:53056 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55', in 0.09 seconds) vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55' vm-test-run-nixbot-gitlab> gitlab # [ 394.431296] nixbot[3426]: INFO: ::ffff:127.0.0.1:53062 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55', in 0.04 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 394.975510] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:56.873Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"70ffb2c6ad3e4a56040fe6c7","created_at":"2026-10-04T11:31:26.097Z","trace_propagation_headers":{"sentry-trace":"fed4a9a5a2824ed5b31f77dfaa2e16ff-9242b7ab92584eda","baggage":"sentry-trace_id=fed4a9a5a2824ed5b31f77dfaa2e16ff,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43AXPXG69BJ0ZNHZBXN0NRM","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:26.131Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-70ffb2c6ad3e4a56040fe6c7: start","job_status":"start","queue_duration_s":90.742089,"scheduling_latency_s":90.742162} vm-test-run-nixbot-gitlab> gitlab # [ 395.211655] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:57.109Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"70ffb2c6ad3e4a56040fe6c7","created_at":"2026-10-04T11:31:26.097Z","trace_propagation_headers":{"sentry-trace":"fed4a9a5a2824ed5b31f77dfaa2e16ff-9242b7ab92584eda","baggage":"sentry-trace_id=fed4a9a5a2824ed5b31f77dfaa2e16ff,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43AXPXG69BJ0ZNHZBXN0NRM","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:26.131Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-70ffb2c6ad3e4a56040fe6c7: done: 0.236966 sec","job_status":"done","queue_duration_s":90.742089,"scheduling_latency_s":90.742162,"redis_calls":23,"redis_duration_s":0.0060160000000000005,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.003158,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001617,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001241,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.131378,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.12924822,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.236966,"completed_at":"2026-10-04T11:32:57.109Z","load_balancing_strategy":"primary","db_duration_s":0.061392,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 395.282029] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:57.112Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"d1b2f3b77b69d8f0bb82bad5","created_at":"2026-10-04T11:31:26.143Z","trace_propagation_headers":{"sentry-trace":"fed4a9a5a2824ed5b31f77dfaa2e16ff-9242b7ab92584eda","baggage":"sentry-trace_id=fed4a9a5a2824ed5b31f77dfaa2e16ff,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43AXPXG69BJ0ZNHZBXN0NRM","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:26.148Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-d1b2f3b77b69d8f0bb82bad5: start","job_status":"start","queue_duration_s":90.963634,"scheduling_latency_s":90.963663} vm-test-run-nixbot-gitlab> gitlab # [ 395.351061] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:57.249Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"d1b2f3b77b69d8f0bb82bad5","created_at":"2026-10-04T11:31:26.143Z","trace_propagation_headers":{"sentry-trace":"fed4a9a5a2824ed5b31f77dfaa2e16ff-9242b7ab92584eda","baggage":"sentry-trace_id=fed4a9a5a2824ed5b31f77dfaa2e16ff,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43AXPXG69BJ0ZNHZBXN0NRM","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:26.148Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-d1b2f3b77b69d8f0bb82bad5: done: 0.137298 sec","job_status":"done","queue_duration_s":90.963634,"scheduling_latency_s":90.963663,"redis_calls":14,"redis_duration_s":0.003272,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000643,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001526,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001103,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.107328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.10604484,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.137298,"completed_at":"2026-10-04T11:32:57.249Z","load_balancing_strategy":"primary","db_duration_s":0.022408,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55' vm-test-run-nixbot-gitlab> gitlab # [ 395.419194] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:57.251Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"d7c522ee45de1802e80c0b94","created_at":"2026-10-04T11:31:26.709Z","trace_propagation_headers":{"sentry-trace":"fed4a9a5a2824ed5b31f77dfaa2e16ff-9242b7ab92584eda","baggage":"sentry-trace_id=fed4a9a5a2824ed5b31f77dfaa2e16ff,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43AXPXG69BJ0ZNHZBXN0NRM","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:26.716Z","job_size_bytes":5,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-d7c522ee45de1802e80c0b94: start","job_status":"start","queue_duration_s":90.535,"scheduling_latency_s":90.535029} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55', in 0.04 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 395.468102] nixbot[3426]: INFO: ::ffff:127.0.0.1:53066 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab # [ 395.554672] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:57.453Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"d7c522ee45de1802e80c0b94","created_at":"2026-10-04T11:31:26.709Z","trace_propagation_headers":{"sentry-trace":"fed4a9a5a2824ed5b31f77dfaa2e16ff-9242b7ab92584eda","baggage":"sentry-trace_id=fed4a9a5a2824ed5b31f77dfaa2e16ff,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43AXPXG69BJ0ZNHZBXN0NRM","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:26.716Z","job_size_bytes":5,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-d7c522ee45de1802e80c0b94: done: 0.201328 sec","job_status":"done","queue_duration_s":90.535,"scheduling_latency_s":90.535029,"redis_calls":12,"redis_duration_s":0.002811,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000803,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001585,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000423,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.166039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030138,"duration_s":0.201328,"completed_at":"2026-10-04T11:32:57.453Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.029387,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 395.625351] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:57.454Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"5cf483624d14314810e5c5ac","created_at":"2026-10-04T11:31:26.747Z","trace_propagation_headers":{"sentry-trace":"fed4a9a5a2824ed5b31f77dfaa2e16ff-9242b7ab92584eda","baggage":"sentry-trace_id=fed4a9a5a2824ed5b31f77dfaa2e16ff,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43AXPXG69BJ0ZNHZBXN0NRM","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:26.753Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-5cf483624d14314810e5c5ac: start","job_status":"start","queue_duration_s":90.701637,"scheduling_latency_s":90.701663} vm-test-run-nixbot-gitlab> gitlab # [ 395.770846] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:57.669Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"5cf483624d14314810e5c5ac","created_at":"2026-10-04T11:31:26.747Z","trace_propagation_headers":{"sentry-trace":"fed4a9a5a2824ed5b31f77dfaa2e16ff-9242b7ab92584eda","baggage":"sentry-trace_id=fed4a9a5a2824ed5b31f77dfaa2e16ff,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43AXPXG69BJ0ZNHZBXN0NRM","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:26.753Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-5cf483624d14314810e5c5ac: done: 0.214717 sec","job_status":"done","queue_duration_s":90.701637,"scheduling_latency_s":90.701663,"redis_calls":11,"redis_duration_s":0.002274,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000554,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001311,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000409,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.176,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.176,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.15368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031498,"duration_s":0.214717,"completed_at":"2026-10-04T11:32:57.669Z","load_balancing_strategy":"primary","db_duration_s":0.063572,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 395.836945] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:57.671Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"d5d8941e84cc5b442dc2ab13","created_at":"2026-10-04T11:31:26.756Z","trace_propagation_headers":{"sentry-trace":"fed4a9a5a2824ed5b31f77dfaa2e16ff-9242b7ab92584eda","baggage":"sentry-trace_id=fed4a9a5a2824ed5b31f77dfaa2e16ff,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43AXPXG69BJ0ZNHZBXN0NRM","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:26.761Z","job_size_bytes":89,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-d5d8941e84cc5b442dc2ab13: start","job_status":"start","queue_duration_s":90.909503,"scheduling_latency_s":90.909522} vm-test-run-nixbot-gitlab> gitlab # [ 395.864836] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:57.708Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"d5d8941e84cc5b442dc2ab13","created_at":"2026-10-04T11:31:26.756Z","trace_propagation_headers":{"sentry-trace":"fed4a9a5a2824ed5b31f77dfaa2e16ff-9242b7ab92584eda","baggage":"sentry-trace_id=fed4a9a5a2824ed5b31f77dfaa2e16ff,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43AXPXG69BJ0ZNHZBXN0NRM","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:26.761Z","job_size_bytes":89,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-d5d8941e84cc5b442dc2ab13: done: 0.037809 sec","job_status":"done","queue_duration_s":90.909503,"scheduling_latency_s":90.909522,"redis_calls":11,"redis_duration_s":0.001933,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000505,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001095,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000333,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027678,"duration_s":0.037809,"completed_at":"2026-10-04T11:32:57.708Z","load_balancing_strategy":"primary","db_duration_s":0.005189,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 395.938352] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:57.710Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 baa7927c76a37407367ce093ce7f4d35363a479f refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"6acedfdd170d1fd583f7614b","created_at":"2026-10-04T11:31:34.620Z","trace_propagation_headers":{"sentry-trace":"c0398371a816422da32c384df5c37679-bf8136436d5a4a5b","baggage":"sentry-trace_id=c0398371a816422da32c384df5c37679,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43AXZBCPGX6HS1MSC6G2F67","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:34.654Z","job_size_bytes":185,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-6acedfdd170d1fd583f7614b: start","job_status":"start","queue_duration_s":83.055334,"scheduling_latency_s":83.055353} vm-test-run-nixbot-gitlab> gitlab # [ 395.962433] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:57.727Z","message":"changes: \"0000000000000000000000000000000000000000 baa7927c76a37407367ce093ce7f4d35363a479f refs/heads/master\\n\""} vm-test-run-nixbot-gitlab> gitlab # [ 395.966006] gitaly[910]: time="2026-10-04T11:32:57.765Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T11:33:07.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:32:57.755" grpc.time_ms=10.192 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 395.982190] gitaly[910]: time="2026-10-04T11:32:57.781Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T11:33:07.021" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:32:57.771" grpc.time_ms=9.546 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 395.998358] gitaly[910]: time="2026-10-04T11:32:57.845Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-04T11:33:07.082" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:32:57.833" grpc.time_ms=11.717 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 396.013872] gitaly[910]: time="2026-10-04T11:32:57.896Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:32:57.895" grpc.time_ms=0.533 pid=910 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 396.025901] gitaly[910]: time="2026-10-04T11:32:57.924Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3939 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:32:57.924" grpc.time_ms=0.513 pid=910 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 396.036863] gitaly[910]: time="2026-10-04T11:32:57.931Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 diskcache=affdc2f3-0798-44f3-87c3-dc0574e479d5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T11:33:07.108" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:32:57.859" pid=910 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 396.047517] gitaly[910]: time="2026-10-04T11:32:57.932Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=77 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5261 command.oublock=8 command.real_time_ms=68 command.spawn_token_fork_ms=0 command.system_time_ms=36 command.user_time_ms=41 component=gitaly.UnaryServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T11:33:07.108" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:32:57.859" grpc.time_ms=72.823 pid=910 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 396.061956] gitaly[910]: time="2026-10-04T11:32:57.947Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T11:33:07.197" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:32:57.947" grpc.time_ms=0.213 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 396.073169] gitaly[910]: time="2026-10-04T11:32:57.966Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=858 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-04T11:33:07.201" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:32:57.952" grpc.time_ms=14.711 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 396.180874] gitaly[910]: time="2026-10-04T11:32:58.079Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=313 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-10-04T11:33:28.072" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:32:58.073" grpc.time_ms=6.639 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 396.217037] gitaly[910]: time="2026-10-04T11:32:58.116Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-04T11:33:08.005" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:32:58.115" grpc.time_ms=0.313 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 396.232782] gitaly[910]: time="2026-10-04T11:32:58.125Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=316 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-04T11:33:08.008" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-10-04T11:32:58.118" grpc.time_ms=6.514 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 396.251823] gitaly[910]: time="2026-10-04T11:32:58.150Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T11:33:08.031" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:32:58.141" grpc.time_ms=8.762 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 396.272515] gitaly[910]: time="2026-10-04T11:32:58.165Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T11:33:08.046" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:32:58.156" grpc.time_ms=9.374 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 396.296590] gitaly[910]: time="2026-10-04T11:32:58.195Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-04T11:33:28.180" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:32:58.181" grpc.time_ms=14.261 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55' vm-test-run-nixbot-gitlab> gitlab # [ 396.507832] nixbot[3426]: INFO: ::ffff:127.0.0.1:53078 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55', in 0.04 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 396.799694] gitaly[910]: time="2026-10-04T11:32:58.698Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-04T11:33:08.006" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:32:58.696" grpc.time_ms=1.706 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 396.832613] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:58.730Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 baa7927c76a37407367ce093ce7f4d35363a479f refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"6acedfdd170d1fd583f7614b","created_at":"2026-10-04T11:31:34.620Z","trace_propagation_headers":{"sentry-trace":"c0398371a816422da32c384df5c37679-bf8136436d5a4a5b","baggage":"sentry-trace_id=c0398371a816422da32c384df5c37679,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43AXZBCPGX6HS1MSC6G2F67","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:31:34.654Z","job_size_bytes":185,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-6acedfdd170d1fd583f7614b: done: 1.019945 sec","job_status":"done","queue_duration_s":83.055334,"scheduling_latency_s":83.055353,"gitaly_calls":13,"gitaly_duration_s":0.204453,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.037951,"redis_read_bytes":2103,"redis_write_bytes":21748,"redis_cache_calls":15,"redis_cache_duration_s":0.003599,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.005231,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001398,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7836,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004133,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.014479,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.009111,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.269,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.301,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.057,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.724041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00069874,"exclusive_lock_hold_duration_s":0.02889972,"duration_s":1.019945,"completed_at":"2026-10-04T11:32:58.730Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.100017,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 396.909284] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:58.732Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"88d56d3d1cb394e56866cf1a","created_at":"2026-10-04T11:31:40.217Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-2f9b9c4ab58d4441","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:40.228Z","job_size_bytes":65,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-88d56d3d1cb394e56866cf1a: start","job_status":"start","queue_duration_s":78.503718,"scheduling_latency_s":78.503738} vm-test-run-nixbot-gitlab> gitlab # [ 396.928519] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:58.816Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"88d56d3d1cb394e56866cf1a","created_at":"2026-10-04T11:31:40.217Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-2f9b9c4ab58d4441","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:40.228Z","job_size_bytes":65,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-88d56d3d1cb394e56866cf1a: done: 0.084113 sec","job_status":"done","queue_duration_s":78.503718,"scheduling_latency_s":78.503738,"redis_calls":16,"redis_duration_s":0.003042,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001605,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001098,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000339,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003397,"duration_s":0.084113,"completed_at":"2026-10-04T11:32:58.816Z","load_balancing_strategy":"primary","db_duration_s":0.021289,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 396.975734] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:58.817Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"dff50f810ae409bbce66764f","created_at":"2026-10-04T11:31:40.229Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-2f9b9c4ab58d4441","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:40.236Z","job_size_bytes":65,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-dff50f810ae409bbce66764f: start","job_status":"start","queue_duration_s":78.581145,"scheduling_latency_s":78.581162} vm-test-run-nixbot-gitlab> gitlab # [ 397.018068] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:58.916Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"dff50f810ae409bbce66764f","created_at":"2026-10-04T11:31:40.229Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-2f9b9c4ab58d4441","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:40.236Z","job_size_bytes":65,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-dff50f810ae409bbce66764f: done: 0.09917 sec","job_status":"done","queue_duration_s":78.581145,"scheduling_latency_s":78.581162,"redis_calls":21,"redis_duration_s":0.004247,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.001759,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001445,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001043,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.061949,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030004,"duration_s":0.09917,"completed_at":"2026-10-04T11:32:58.916Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.029301,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 397.068620] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:58.918Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"d10d7d6f85518a067ebf2313","created_at":"2026-10-04T11:31:40.393Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-2f9b9c4ab58d4441","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:40.400Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d10d7d6f85518a067ebf2313: start","job_status":"start","queue_duration_s":78.518104,"scheduling_latency_s":78.518123} vm-test-run-nixbot-gitlab> gitlab # [ 397.340778] gitaly[910]: time="2026-10-04T11:32:59.239Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43AY4VG20322ADMX7R8413B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:32:59.238" grpc.time_ms=0.518 pid=910 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 397.371876] gitaly[910]: time="2026-10-04T11:32:59.270Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43AY4VG20322ADMX7R8413B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:32:59.270" grpc.time_ms=0.499 pid=910 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 397.401957] gitaly[910]: time="2026-10-04T11:32:59.300Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43AY4VG20322ADMX7R8413B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:32:59.300" grpc.time_ms=0.468 pid=910 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 397.416596] gitaly[910]: time="2026-10-04T11:32:59.310Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M43AY4VG20322ADMX7R8413B diskcache=c71d941a-490b-4bff-92c7-0d2ca9f06695 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T11:33:29.204" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:32:59.205" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 397.430142] gitaly[910]: time="2026-10-04T11:32:59.310Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=116 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7524 command.oublock=0 command.real_time_ms=100 command.spawn_token_fork_ms=0 command.system_time_ms=57 command.user_time_ms=59 component=gitaly.UnaryServerInterceptor correlation_id=01M43AY4VG20322ADMX7R8413B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T11:33:29.204" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:32:59.205" grpc.time_ms=104.774 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55' vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55', in 0.04 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 397.546238] nixbot[3426]: INFO: ::ffff:127.0.0.1:53084 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab # [ 397.550526] gitaly[910]: time="2026-10-04T11:32:59.449Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43AY4VG20322ADMX7R8413B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T11:33:29.448" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:32:59.448" grpc.time_ms=1.02 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 397.591046] gitaly[910]: time="2026-10-04T11:32:59.489Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AY4VG20322ADMX7R8413B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:29.488" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:32:59.488" grpc.time_ms=0.823 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 397.611494] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:59.507Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"d10d7d6f85518a067ebf2313","created_at":"2026-10-04T11:31:40.393Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-2f9b9c4ab58d4441","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T11:31:40.400Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d10d7d6f85518a067ebf2313: done: 0.589466 sec","job_status":"done","queue_duration_s":78.518104,"scheduling_latency_s":78.518123,"gitaly_calls":3,"gitaly_duration_s":0.113774,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.025811,"redis_read_bytes":455,"redis_write_bytes":27999,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001589,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001159,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.008767,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.003922,"redis_queues_read_bytes":34,"redis_queues_write_bytes":14494,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.004506,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000713,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":165,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001603,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001963,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.062,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.024,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.027,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.032,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.392276,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.52707726,"duration_s":0.589466,"completed_at":"2026-10-04T11:32:59.507Z","load_balancing_strategy":"primary","db_duration_s":0.072918,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 397.679594] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:59.508Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","baa7927c76a37407367ce093ce7f4d35363a479f b1c994f1551ecfb35328e5078741a48f638cbb55 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"564541e4fc6548b89a367bcd","created_at":"2026-10-04T11:32:55.087Z","trace_propagation_headers":{"sentry-trace":"0c83733bdf8a48ca960e4ea6962d1d36-8b99cf78743c4482","baggage":"sentry-trace_id=0c83733bdf8a48ca960e4ea6962d1d36,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43B0E52BK28RD02GWGHJ25S","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:32:55.092Z","job_size_bytes":185,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-564541e4fc6548b89a367bcd: start","job_status":"start","queue_duration_s":4.416458,"scheduling_latency_s":4.416475} vm-test-run-nixbot-gitlab> gitlab # [ 397.697607] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:59.520Z","message":"changes: \"baa7927c76a37407367ce093ce7f4d35363a479f b1c994f1551ecfb35328e5078741a48f638cbb55 refs/heads/master\\n\""} vm-test-run-nixbot-gitlab> gitlab # [ 397.700054] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:59.565Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"e49fd88f5448ff2206f8dfb7","created_at":"2026-10-04T11:32:59.563Z","trace_propagation_headers":{"sentry-trace":"0c83733bdf8a48ca960e4ea6962d1d36-b9e8224d176449db","baggage":"sentry-trace_id=0c83733bdf8a48ca960e4ea6962d1d36,sentry-release=Unknown"},"correlation_id":"01M43B0E52BK28RD02GWGHJ25S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"f724645df6111d9559d7c2db","job_size_bytes":46,"pid":3304,"job_status":"deduplicated","message":"ProjectCacheWorker JID-e49fd88f5448ff2206f8dfb7: deduplicated: until executing","deduplication.type":"until executing"} vm-test-run-nixbot-gitlab> gitlab # [ 397.715383] gitaly[910]: time="2026-10-04T11:32:59.547Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T11:33:09.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:32:59.537" grpc.time_ms=10.017 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 397.729349] gitaly[910]: time="2026-10-04T11:32:59.561Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T11:33:09.021" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:32:59.551" grpc.time_ms=9.903 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 397.743137] gitaly[910]: time="2026-10-04T11:32:59.586Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=8 component=gitaly.StreamServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-04T11:33:09.045" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:32:59.576" grpc.time_ms=10.603 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 397.757384] gitaly[910]: time="2026-10-04T11:32:59.602Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=594 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=8 component=gitaly.StreamServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-10-04T11:33:29.591" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:32:59.591" grpc.time_ms=11.112 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 397.774551] gitaly[910]: time="2026-10-04T11:32:59.621Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=319 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-04T11:33:09.084" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-10-04T11:32:59.614" grpc.time_ms=6.541 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 397.788714] gitaly[910]: time="2026-10-04T11:32:59.640Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T11:33:09.100" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:32:59.630" grpc.time_ms=9.936 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 397.850787] gitaly[910]: time="2026-10-04T11:32:59.749Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-04T11:33:09.057" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:32:59.748" grpc.time_ms=1.427 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 397.874878] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:59.772Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","baa7927c76a37407367ce093ce7f4d35363a479f b1c994f1551ecfb35328e5078741a48f638cbb55 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"564541e4fc6548b89a367bcd","created_at":"2026-10-04T11:32:55.087Z","trace_propagation_headers":{"sentry-trace":"0c83733bdf8a48ca960e4ea6962d1d36-8b99cf78743c4482","baggage":"sentry-trace_id=0c83733bdf8a48ca960e4ea6962d1d36,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43B0E52BK28RD02GWGHJ25S","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:32:55.092Z","job_size_bytes":185,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-564541e4fc6548b89a367bcd: done: 0.263702 sec","job_status":"done","queue_duration_s":4.416458,"scheduling_latency_s":4.416475,"gitaly_calls":7,"gitaly_duration_s":0.074889,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.019377,"redis_read_bytes":501,"redis_write_bytes":11943,"redis_cache_calls":9,"redis_cache_duration_s":0.003196,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001024,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5720,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003113,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.0056,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.006444,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.027,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.027,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.162444,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00059892,"exclusive_lock_hold_duration_s":0.01823052,"duration_s":0.263702,"completed_at":"2026-10-04T11:32:59.772Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026804,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 397.942829] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:59.775Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"f724645df6111d9559d7c2db","created_at":"2026-10-04T11:32:57.784Z","trace_propagation_headers":{"sentry-trace":"c0398371a816422da32c384df5c37679-260e1ffd40b446f9","baggage":"sentry-trace_id=c0398371a816422da32c384df5c37679,sentry-release=Unknown"},"correlation_id":"01M43AXZBCPGX6HS1MSC6G2F67","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:57.803Z","job_size_bytes":46,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-f724645df6111d9559d7c2db: start","job_status":"start","queue_duration_s":1.971486,"scheduling_latency_s":1.971515} vm-test-run-nixbot-gitlab> gitlab # [ 397.959500] gitaly[910]: time="2026-10-04T11:32:59.842Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-10-04T17:32:59.836" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:32:59.837" grpc.time_ms=5.275 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 397.970970] gitaly[910]: time="2026-10-04T11:32:59.863Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-04T11:33:29.852" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:32:59.853" grpc.time_ms=9.964 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 397.994585] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:59.892Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"f724645df6111d9559d7c2db","created_at":"2026-10-04T11:32:57.784Z","trace_propagation_headers":{"sentry-trace":"c0398371a816422da32c384df5c37679-260e1ffd40b446f9","baggage":"sentry-trace_id=c0398371a816422da32c384df5c37679,sentry-release=Unknown"},"correlation_id":"01M43AXZBCPGX6HS1MSC6G2F67","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:57.803Z","job_size_bytes":46,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-f724645df6111d9559d7c2db: done: 0.117481 sec","job_status":"done","queue_duration_s":1.971486,"scheduling_latency_s":1.971515,"gitaly_calls":2,"gitaly_duration_s":0.019977,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007023,"redis_read_bytes":69,"redis_write_bytes":4888,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001254,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000513,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001578,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002955,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000723,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.078748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033634,"duration_s":0.117481,"completed_at":"2026-10-04T11:32:59.892Z","load_balancing_strategy":"primary","db_duration_s":0.012464,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 398.049802] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:59.894Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"af1a7f83800993499ac60d7a","created_at":"2026-10-04T11:32:57.821Z","trace_propagation_headers":{"sentry-trace":"c0398371a816422da32c384df5c37679-260e1ffd40b446f9","baggage":"sentry-trace_id=c0398371a816422da32c384df5c37679,sentry-release=Unknown"},"correlation_id":"01M43AXZBCPGX6HS1MSC6G2F67","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:57.828Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-af1a7f83800993499ac60d7a: start","job_status":"start","queue_duration_s":2.066355,"scheduling_latency_s":2.066376} vm-test-run-nixbot-gitlab> gitlab # [ 398.067153] gitaly[910]: time="2026-10-04T11:32:59.941Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 diskcache=d319e3de-4e77-4482-9f84-8b4b0c41dba4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-04T17:32:59.919" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:32:59.919" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 398.077976] gitaly[910]: time="2026-10-04T11:32:59.942Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1174 command.oublock=0 command.real_time_ms=27 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-04T17:32:59.919" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:32:59.919" grpc.time_ms=22.089 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 398.093918] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:59.981Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"af1a7f83800993499ac60d7a","created_at":"2026-10-04T11:32:57.821Z","trace_propagation_headers":{"sentry-trace":"c0398371a816422da32c384df5c37679-260e1ffd40b446f9","baggage":"sentry-trace_id=c0398371a816422da32c384df5c37679,sentry-release=Unknown"},"correlation_id":"01M43AXZBCPGX6HS1MSC6G2F67","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:57.828Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-af1a7f83800993499ac60d7a: done: 0.086891 sec","job_status":"done","queue_duration_s":2.066355,"scheduling_latency_s":2.066376,"gitaly_calls":1,"gitaly_duration_s":0.024483,"redis_calls":15,"redis_duration_s":0.003057,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000397,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001158,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000492,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00101,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.011,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.021,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04225,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.06193982,"duration_s":0.086891,"completed_at":"2026-10-04T11:32:59.981Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015728,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 398.142054] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:32:59.982Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"9d0b2e7cc7eac846ea73452b","created_at":"2026-10-04T11:32:57.942Z","trace_propagation_headers":{"sentry-trace":"c0398371a816422da32c384df5c37679-260e1ffd40b446f9","baggage":"sentry-trace_id=c0398371a816422da32c384df5c37679,sentry-release=Unknown"},"correlation_id":"01M43AXZBCPGX6HS1MSC6G2F67","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:57.944Z","job_size_bytes":23,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-9d0b2e7cc7eac846ea73452b: start","job_status":"start","queue_duration_s":2.038393,"scheduling_latency_s":2.038411} vm-test-run-nixbot-gitlab> gitlab # [ 398.158144] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:00.029Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"9d0b2e7cc7eac846ea73452b","created_at":"2026-10-04T11:32:57.942Z","trace_propagation_headers":{"sentry-trace":"c0398371a816422da32c384df5c37679-260e1ffd40b446f9","baggage":"sentry-trace_id=c0398371a816422da32c384df5c37679,sentry-release=Unknown"},"correlation_id":"01M43AXZBCPGX6HS1MSC6G2F67","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:57.944Z","job_size_bytes":23,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-9d0b2e7cc7eac846ea73452b: done: 0.046719 sec","job_status":"done","queue_duration_s":2.038393,"scheduling_latency_s":2.038411,"gitaly_calls":1,"gitaly_duration_s":0.011513,"redis_calls":16,"redis_duration_s":0.003303,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000256,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001528,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000616,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046719,"completed_at":"2026-10-04T11:33:00.029Z","load_balancing_strategy":"primary","db_duration_s":0.003124,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 398.205620] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:00.031Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"9d0a3c4ee13ba513369a7f80","created_at":"2026-10-04T11:32:58.520Z","trace_propagation_headers":{"sentry-trace":"c0398371a816422da32c384df5c37679-260e1ffd40b446f9","baggage":"sentry-trace_id=c0398371a816422da32c384df5c37679,sentry-release=Unknown"},"correlation_id":"01M43AXZBCPGX6HS1MSC6G2F67","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:624db244e3471950d4a0ff39246a4ebd816a72a7026346f2d82aa7274acac18d","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:58.525Z","job_size_bytes":467,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-9d0a3c4ee13ba513369a7f80: start","job_status":"start","queue_duration_s":1.505236,"scheduling_latency_s":1.505254} vm-test-run-nixbot-gitlab> gitlab # [ 398.223427] gitaly[910]: time="2026-10-04T11:33:00.020Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-04T11:33:30.010" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:00.010" grpc.time_ms=9.598 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 398.237017] gitaly[910]: time="2026-10-04T11:33:00.058Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:30.057" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:00.057" grpc.time_ms=0.751 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 398.249829] gitaly[910]: time="2026-10-04T11:33:00.069Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-04T11:33:10.000" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:33:00.060" grpc.time_ms=8.748 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 398.295937] gitaly[910]: time="2026-10-04T11:33:00.194Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T11:33:30.093" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:00.193" grpc.time_ms=0.918 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 398.316400] gitaly[910]: time="2026-10-04T11:33:00.201Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=33 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-04T11:33:30.098" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:00.198" grpc.time_ms=2.56 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 398.401344] gitaly[910]: time="2026-10-04T11:33:00.299Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T11:33:10.229" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:33:00.289" grpc.time_ms=10.023 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55' vm-test-run-nixbot-gitlab> gitlab # [ 398.590675] nixbot[3426]: INFO: ::ffff:127.0.0.1:53088 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55', in 0.05 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 399.177518] gitaly[910]: time="2026-10-04T11:33:01.076Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-04T11:33:31.058" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:01.059" grpc.time_ms=16.974 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 399.202195] gitaly[910]: time="2026-10-04T11:33:01.093Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M43AXZBCPGX6HS1MSC6G2F67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-04T11:33:31.079" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:01.079" grpc.time_ms=14.062 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 399.235480] sidekiq-cluster[3304]: {"severity":"WARN","time":"2026-10-04T11:33:01.133Z","class":"CreatePipelineWorker","correlation_id":"01M43AXZBCPGX6HS1MSC6G2F67","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"baa7927c76a37407367ce093ce7f4d35363a479f","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"baa7927c76a37407367ce093ce7f4d35363a479f","ref":"refs/heads/master"},"retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 399.247322] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.144Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"9d0a3c4ee13ba513369a7f80","created_at":"2026-10-04T11:32:58.520Z","trace_propagation_headers":{"sentry-trace":"c0398371a816422da32c384df5c37679-260e1ffd40b446f9","baggage":"sentry-trace_id=c0398371a816422da32c384df5c37679,sentry-release=Unknown"},"correlation_id":"01M43AXZBCPGX6HS1MSC6G2F67","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:624db244e3471950d4a0ff39246a4ebd816a72a7026346f2d82aa7274acac18d","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:58.525Z","job_size_bytes":467,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-9d0a3c4ee13ba513369a7f80: done: 1.113035 sec","job_status":"done","queue_duration_s":1.505236,"scheduling_latency_s":1.505254,"gitaly_calls":7,"gitaly_duration_s":0.069954,"redis_calls":51,"redis_duration_s":0.009385000000000001,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.000812,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.004007,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001165,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.00236,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.989135,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035746,"duration_s":1.113035,"completed_at":"2026-10-04T11:33:01.143Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.056912,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 399.310944] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.146Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"f7f34050-a5ed-4dda-8d6c-130b8172c527\"}"],"class":"WebHookWorker","jid":"56fac43511d059ec041be86b","created_at":"2026-10-04T11:32:58.663Z","trace_propagation_headers":{"sentry-trace":"c0398371a816422da32c384df5c37679-260e1ffd40b446f9","baggage":"sentry-trace_id=c0398371a816422da32c384df5c37679,sentry-release=Unknown"},"correlation_id":"01M43AXZBCPGX6HS1MSC6G2F67","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:29217b8a48e7e2e6728ddda1b3b45e6e796a5151261fac201395f23e1ff9b6ea","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:58.668Z","job_size_bytes":1751,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-56fac43511d059ec041be86b: start","job_status":"start","queue_duration_s":2.478041,"scheduling_latency_s":2.478069} vm-test-run-nixbot-gitlab> gitlab # [ 399.414366] nixbot[3426]: INFO: ::ffff:127.0.0.1:53104 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted vm-test-run-nixbot-gitlab> gitlab # [ 399.440211] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.337Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"f7f34050-a5ed-4dda-8d6c-130b8172c527\"}"],"class":"WebHookWorker","jid":"56fac43511d059ec041be86b","created_at":"2026-10-04T11:32:58.663Z","trace_propagation_headers":{"sentry-trace":"c0398371a816422da32c384df5c37679-260e1ffd40b446f9","baggage":"sentry-trace_id=c0398371a816422da32c384df5c37679,sentry-release=Unknown"},"correlation_id":"01M43AXZBCPGX6HS1MSC6G2F67","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:29217b8a48e7e2e6728ddda1b3b45e6e796a5151261fac201395f23e1ff9b6ea","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:58.668Z","job_size_bytes":1751,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-56fac43511d059ec041be86b: done: 0.191145 sec","job_status":"done","queue_duration_s":2.478041,"scheduling_latency_s":2.478069,"redis_calls":19,"redis_duration_s":0.00511,"redis_read_bytes":32,"redis_write_bytes":5367,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001241,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000518,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001937,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001414,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.06695844000000761,"cpu_s":0.109057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00055214,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.191145,"completed_at":"2026-10-04T11:33:01.337Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004295,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 399.506889] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.340Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f3deed46a5f0676b4661f05b","created_at":"2026-10-04T11:32:59.188Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.193Z","job_size_bytes":23,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f3deed46a5f0676b4661f05b: start","job_status":"start","queue_duration_s":2.147208,"scheduling_latency_s":2.147235} vm-test-run-nixbot-gitlab> gitlab # [ 399.531716] gitaly[910]: time="2026-10-04T11:33:01.430Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M43B0MRX4G6PHSE877XN5N2J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:01.419" grpc.time_ms=10.572 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 399.545954] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:01 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 90 vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55' vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55', in 0.04 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 399.583386] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.481Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f3deed46a5f0676b4661f05b","created_at":"2026-10-04T11:32:59.188Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.193Z","job_size_bytes":23,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f3deed46a5f0676b4661f05b: done: 0.141045 sec","job_status":"done","queue_duration_s":2.147208,"scheduling_latency_s":2.147235,"redis_calls":10,"redis_duration_s":0.0028480000000000003,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000448,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001832,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000568,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.095233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040802,"duration_s":0.141045,"completed_at":"2026-10-04T11:33:01.481Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.035169,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 399.644985] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.483Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"24f1e5b9fc0c54fcfe775319","created_at":"2026-10-04T11:32:59.316Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.322Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-24f1e5b9fc0c54fcfe775319: start","job_status":"start","queue_duration_s":2.160973,"scheduling_latency_s":2.160995} vm-test-run-nixbot-gitlab> gitlab # [ 399.664163] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.542Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"24f1e5b9fc0c54fcfe775319","created_at":"2026-10-04T11:32:59.316Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.322Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-24f1e5b9fc0c54fcfe775319: done: 0.059368 sec","job_status":"done","queue_duration_s":2.160973,"scheduling_latency_s":2.160995,"redis_calls":10,"redis_duration_s":0.00466,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001122,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003079,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000459,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032882,"duration_s":0.059368,"completed_at":"2026-10-04T11:33:01.542Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011032,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 399.710657] gitaly[910]: time="2026-10-04T11:33:01.522Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M43B0MVSJF5CJR7TGTP7G5BF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:01.512" pid=910 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 399.721830] nixbot[3426]: INFO: ::ffff:127.0.0.1:53112 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab # [ 399.724234] gitlab-workhorse[817]: time="2026-10-04T11:33:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 399.726155] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 90 vm-test-run-nixbot-gitlab> gitlab # [ 399.728142] gitlab-workhorse[817]: time="2026-10-04T11:33:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=336 vm-test-run-nixbot-gitlab> gitlab # [ 399.730056] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 336 "" "git/2.55.0" 83 vm-test-run-nixbot-gitlab> gitlab # [ 399.732048] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.544Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"e989837f1837e926878c67bf","created_at":"2026-10-04T11:32:59.341Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.347Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-e989837f1837e926878c67bf: start","job_status":"start","queue_duration_s":2.197222,"scheduling_latency_s":2.197244} vm-test-run-nixbot-gitlab> gitlab # [ 399.748473] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.604Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"e989837f1837e926878c67bf","created_at":"2026-10-04T11:32:59.341Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.347Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-e989837f1837e926878c67bf: done: 0.0601 sec","job_status":"done","queue_duration_s":2.197222,"scheduling_latency_s":2.197244,"redis_calls":10,"redis_duration_s":0.002429,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000402,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001484,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000543,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039724,"duration_s":0.0601,"completed_at":"2026-10-04T11:33:01.604Z","load_balancing_strategy":"primary","db_duration_s":0.013179,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 399.793981] gitaly[910]: time="2026-10-04T11:33:01.522Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=4 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0MVSJF5CJR7TGTP7G5BF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:01.512" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 399.808100] nixbot[3426]: {"ts": "2026-10-04T11:33:01.650Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} vm-test-run-nixbot-gitlab> gitlab # [ 399.810592] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.608Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"bb00bb7828a49c8219f37b12","created_at":"2026-10-04T11:32:59.366Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.373Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-bb00bb7828a49c8219f37b12: start","job_status":"start","queue_duration_s":2.234841,"scheduling_latency_s":2.234868} vm-test-run-nixbot-gitlab> gitlab # [ 399.827771] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.665Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"bb00bb7828a49c8219f37b12","created_at":"2026-10-04T11:32:59.366Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.373Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-bb00bb7828a49c8219f37b12: done: 0.057829 sec","job_status":"done","queue_duration_s":2.234841,"scheduling_latency_s":2.234868,"redis_calls":13,"redis_duration_s":0.004742,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000464,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001582,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002108,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000588,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004794,"duration_s":0.057829,"completed_at":"2026-10-04T11:33:01.665Z","load_balancing_strategy":"primary","db_duration_s":0.008495,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 399.875788] gitaly[910]: time="2026-10-04T11:33:01.523Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=4 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0MVSJF5CJR7TGTP7G5BF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:01.512" grpc.time_ms=10.635 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 399.890980] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.667Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"3197a0e2602f3adfa4a38164","created_at":"2026-10-04T11:32:59.373Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.379Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-3197a0e2602f3adfa4a38164: start","job_status":"start","queue_duration_s":2.288342,"scheduling_latency_s":2.288368} vm-test-run-nixbot-gitlab> gitlab # [ 399.908592] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.711Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"3197a0e2602f3adfa4a38164","created_at":"2026-10-04T11:32:59.373Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.379Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-3197a0e2602f3adfa4a38164: done: 0.043943 sec","job_status":"done","queue_duration_s":2.288342,"scheduling_latency_s":2.288368,"redis_calls":10,"redis_duration_s":0.0020369999999999997,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000381,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001253,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000403,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029821,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036842,"duration_s":0.043943,"completed_at":"2026-10-04T11:33:01.711Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005656,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 399.954961] gitaly[910]: time="2026-10-04T11:33:01.608Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M43B0MYPMPWJJ5EXZAJVSRXX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:01.592" pid=910 remote_ip="2001:db8:1::1" request_sha=eb530f36d5f3e6a4e06bf6048042c521026f2c68 response_bytes=336 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 399.965844] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.714Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"4fe3827916c2487360e931c8","created_at":"2026-10-04T11:32:59.380Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.386Z","job_size_bytes":19,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-4fe3827916c2487360e931c8: start","job_status":"start","queue_duration_s":2.328026,"scheduling_latency_s":2.328055} vm-test-run-nixbot-gitlab> gitlab # [ 399.982678] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.765Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"4fe3827916c2487360e931c8","created_at":"2026-10-04T11:32:59.380Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.386Z","job_size_bytes":19,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-4fe3827916c2487360e931c8: done: 0.051467 sec","job_status":"done","queue_duration_s":2.328026,"scheduling_latency_s":2.328055,"redis_calls":11,"redis_duration_s":0.007170999999999999,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0014,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.004729,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00047818,"duration_s":0.051467,"completed_at":"2026-10-04T11:33:01.765Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007571,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 400.028982] gitaly[910]: time="2026-10-04T11:33:01.608Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1130 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0MYPMPWJJ5EXZAJVSRXX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:01.592" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 400.044269] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.767Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ccaf4ca64c1adfd093d467a0","created_at":"2026-10-04T11:32:59.390Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.395Z","job_size_bytes":66,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ccaf4ca64c1adfd093d467a0: start","job_status":"start","queue_duration_s":2.372391,"scheduling_latency_s":2.372419} vm-test-run-nixbot-gitlab> gitlab # [ 400.062363] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.814Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ccaf4ca64c1adfd093d467a0","created_at":"2026-10-04T11:32:59.390Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.395Z","job_size_bytes":66,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ccaf4ca64c1adfd093d467a0: done: 0.046975 sec","job_status":"done","queue_duration_s":2.372391,"scheduling_latency_s":2.372419,"redis_calls":10,"redis_duration_s":0.00417,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000449,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003255,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000466,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034694,"duration_s":0.046975,"completed_at":"2026-10-04T11:33:01.814Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00618,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 400.110139] gitaly[910]: time="2026-10-04T11:33:01.609Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1130 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0MYPMPWJJ5EXZAJVSRXX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:01.592" grpc.time_ms=16.555 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 400.125666] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.816Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"f1588fd023dc2c61c550c0cd","created_at":"2026-10-04T11:32:59.396Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.401Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-f1588fd023dc2c61c550c0cd: start","job_status":"start","queue_duration_s":2.415045,"scheduling_latency_s":2.415063} vm-test-run-nixbot-gitlab> gitlab # [ 400.142584] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.879Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"f1588fd023dc2c61c550c0cd","created_at":"2026-10-04T11:32:59.396Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.401Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-f1588fd023dc2c61c550c0cd: done: 0.062942 sec","job_status":"done","queue_duration_s":2.415045,"scheduling_latency_s":2.415063,"redis_calls":10,"redis_duration_s":0.002464,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000499,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001501,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000464,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030882,"duration_s":0.062942,"completed_at":"2026-10-04T11:33:01.879Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007037,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 400.188655] gitaly[910]: time="2026-10-04T11:33:01.760Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0N3271P9HTAFRP4AYZGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:31.459" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:01.760" grpc.time_ms=0.624 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 400.201819] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.881Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"8ac49933796964b40789027f","created_at":"2026-10-04T11:32:59.409Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.414Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-8ac49933796964b40789027f: start","job_status":"start","queue_duration_s":2.46668,"scheduling_latency_s":2.466701} vm-test-run-nixbot-gitlab> gitlab # [ 400.218958] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.919Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"8ac49933796964b40789027f","created_at":"2026-10-04T11:32:59.409Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.414Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-8ac49933796964b40789027f: done: 0.038906 sec","job_status":"done","queue_duration_s":2.46668,"scheduling_latency_s":2.466701,"redis_calls":10,"redis_duration_s":0.002153,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000459,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001238,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000456,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023412,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00041792,"duration_s":0.038906,"completed_at":"2026-10-04T11:33:01.919Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00962,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 400.266201] gitaly[910]: time="2026-10-04T11:33:01.916Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M43B0N3271P9HTAFRP4AYZGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T11:33:11.098" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:33:01.908" grpc.time_ms=8.117 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 400.280241] gitaly[910]: time="2026-10-04T11:33:02.074Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S diskcache=c108b6c1-126d-41d8-bddf-2e98736ed1f7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-04T17:33:02.066" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:02.067" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 400.290863] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.921Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3dc4eea0e86362d728b2b617","created_at":"2026-10-04T11:32:59.414Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.419Z","job_size_bytes":6,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3dc4eea0e86362d728b2b617: start","job_status":"start","queue_duration_s":2.50165,"scheduling_latency_s":2.501666} vm-test-run-nixbot-gitlab> gitlab # [ 400.307404] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.945Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3dc4eea0e86362d728b2b617","created_at":"2026-10-04T11:32:59.414Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.419Z","job_size_bytes":6,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3dc4eea0e86362d728b2b617: done: 0.024449 sec","job_status":"done","queue_duration_s":2.50165,"scheduling_latency_s":2.501666,"redis_calls":10,"redis_duration_s":0.001958,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000394,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001145,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000419,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035844,"duration_s":0.024449,"completed_at":"2026-10-04T11:33:01.945Z","load_balancing_strategy":"primary","db_duration_s":0.002611,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 400.353191] gitaly[910]: time="2026-10-04T11:33:02.074Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=290 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-04T17:33:02.066" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:02.067" grpc.time_ms=7.455 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 400.366889] nixbot[3426]: {"ts": "2026-10-04T11:33:02.222Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/baa7927c76a37407367ce093ce7f4d35363a479f \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 400.369942] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.946Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"749543dcd94c4a3bd021be36","created_at":"2026-10-04T11:32:59.420Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.424Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-749543dcd94c4a3bd021be36: start","job_status":"start","queue_duration_s":2.521912,"scheduling_latency_s":2.521929} vm-test-run-nixbot-gitlab> gitlab # [ 400.386305] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.983Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"749543dcd94c4a3bd021be36","created_at":"2026-10-04T11:32:59.420Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.424Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-749543dcd94c4a3bd021be36: done: 0.036732 sec","job_status":"done","queue_duration_s":2.521912,"scheduling_latency_s":2.521929,"redis_calls":10,"redis_duration_s":0.002172,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000358,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001446,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000368,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028876,"duration_s":0.036732,"completed_at":"2026-10-04T11:33:01.983Z","load_balancing_strategy":"primary","db_duration_s":0.006973,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 400.430816] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/baa7927c76a37407367ce093ce7f4d35363a479f HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 554 vm-test-run-nixbot-gitlab> gitlab # [ 400.433555] gitaly[910]: time="2026-10-04T11:33:02.115Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:32.013" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:02.114" grpc.time_ms=0.932 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 400.447652] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:01.984Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"7563deeabeafffa4a2e21277","created_at":"2026-10-04T11:32:59.432Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.436Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-7563deeabeafffa4a2e21277: start","job_status":"start","queue_duration_s":2.54796,"scheduling_latency_s":2.547976} vm-test-run-nixbot-gitlab> gitlab # [ 400.464498] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:02.047Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"7563deeabeafffa4a2e21277","created_at":"2026-10-04T11:32:59.432Z","trace_propagation_headers":{"sentry-trace":"eb66587e2d7741f38055324e3532db63-ac2a1f57d0af4811","baggage":"sentry-trace_id=eb66587e2d7741f38055324e3532db63,sentry-release=Unknown"},"correlation_id":"01M43AY4VG20322ADMX7R8413B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.436Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-7563deeabeafffa4a2e21277: done: 0.062953 sec","job_status":"done","queue_duration_s":2.54796,"scheduling_latency_s":2.547976,"redis_calls":14,"redis_duration_s":0.003981,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000342,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002492,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001147,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043374,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00065866,"duration_s":0.062953,"completed_at":"2026-10-04T11:33:02.047Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011675,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 400.510368] gitaly[910]: time="2026-10-04T11:33:02.124Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-04T11:33:12.057" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:33:02.117" grpc.time_ms=6.872 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 400.523989] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:02.049Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"0e2ff430dcc490e7c24a5bfd","created_at":"2026-10-04T11:32:59.570Z","trace_propagation_headers":{"sentry-trace":"0c83733bdf8a48ca960e4ea6962d1d36-b9e8224d176449db","baggage":"sentry-trace_id=0c83733bdf8a48ca960e4ea6962d1d36,sentry-release=Unknown"},"correlation_id":"01M43B0E52BK28RD02GWGHJ25S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"af1a7f83800993499ac60d7a","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.572Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-0e2ff430dcc490e7c24a5bfd: start","job_status":"start","queue_duration_s":2.47611,"scheduling_latency_s":2.476127} vm-test-run-nixbot-gitlab> gitlab # [ 400.541492] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:02.090Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"0e2ff430dcc490e7c24a5bfd","created_at":"2026-10-04T11:32:59.570Z","trace_propagation_headers":{"sentry-trace":"0c83733bdf8a48ca960e4ea6962d1d36-b9e8224d176449db","baggage":"sentry-trace_id=0c83733bdf8a48ca960e4ea6962d1d36,sentry-release=Unknown"},"correlation_id":"01M43B0E52BK28RD02GWGHJ25S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"af1a7f83800993499ac60d7a","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.572Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-0e2ff430dcc490e7c24a5bfd: done: 0.041519 sec","job_status":"done","queue_duration_s":2.47611,"scheduling_latency_s":2.476127,"gitaly_calls":1,"gitaly_duration_s":0.010166,"redis_calls":12,"redis_duration_s":0.002603,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000477,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001181,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.02382228,"duration_s":0.041519,"completed_at":"2026-10-04T11:33:02.090Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003678,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 400.588995] nixbot[3426]: {"ts": "2026-10-04T11:33:02.392Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/baa7927c76a37407367ce093ce7f4d35363a479f \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 400.592287] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/baa7927c76a37407367ce093ce7f4d35363a479f HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 163 vm-test-run-nixbot-gitlab> gitlab # [ 400.595132] gitaly[910]: time="2026-10-04T11:33:02.160Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T11:33:32.058" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:02.158" grpc.time_ms=1.468 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 400.608544] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:02.091Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"0b57dc3d45922275f80fd1fb","created_at":"2026-10-04T11:32:59.699Z","trace_propagation_headers":{"sentry-trace":"0c83733bdf8a48ca960e4ea6962d1d36-b9e8224d176449db","baggage":"sentry-trace_id=0c83733bdf8a48ca960e4ea6962d1d36,sentry-release=Unknown"},"correlation_id":"01M43B0E52BK28RD02GWGHJ25S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f9c80bd82e479a17b7624aad23d3c004bbd5af740f55bd7a0303df1c2b307d0a","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.702Z","job_size_bytes":467,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-0b57dc3d45922275f80fd1fb: start","job_status":"start","queue_duration_s":2.389782,"scheduling_latency_s":2.389798} vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55' vm-test-run-nixbot-gitlab> gitlab # [ 400.626791] gitaly[910]: time="2026-10-04T11:33:02.168Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=28 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-04T11:33:32.063" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:02.164" grpc.time_ms=3.864 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 400.643175] gitaly[910]: time="2026-10-04T11:33:02.200Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T11:33:12.133" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:33:02.193" grpc.time_ms=6.814 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 400.657016] gitaly[910]: time="2026-10-04T11:33:02.290Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0NMK4GQDSH7Z2269BH90 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:32.088" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:02.289" grpc.time_ms=0.647 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 400.670425] gitaly[910]: time="2026-10-04T11:33:02.334Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M43B0NMK4GQDSH7Z2269BH90 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T11:33:12.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:33:02.327" grpc.time_ms=7.106 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 400.684541] gitaly[910]: time="2026-10-04T11:33:02.456Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0NSXYSSBM5SDGCKV84WW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:32.254" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:02.455" grpc.time_ms=0.821 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 400.698982] nixbot[3426]: {"ts": "2026-10-04T11:33:02.582Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/baa7927c76a37407367ce093ce7f4d35363a479f \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55', in 0.07 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 400.702049] nixbot[3426]: INFO: ::ffff:127.0.0.1:53116 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab # [ 400.703992] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/baa7927c76a37407367ce093ce7f4d35363a479f HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 182 vm-test-run-nixbot-gitlab> gitlab # [ 400.706582] gitaly[910]: time="2026-10-04T11:33:02.524Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M43B0NSXYSSBM5SDGCKV84WW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T11:33:12.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:33:02.516" grpc.time_ms=7.575 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 400.768732] gitaly[910]: time="2026-10-04T11:33:02.667Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M43B0P0ME40A25T8C4A217KE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:02.659" grpc.time_ms=8.543 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 400.783713] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:02 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 55 vm-test-run-nixbot-gitlab> gitlab # [ 400.831321] gitaly[910]: time="2026-10-04T11:33:02.730Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M43B0P2DX1A2WHX38CS2YA0W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:02.719" pid=910 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 400.842693] gitaly[910]: time="2026-10-04T11:33:02.730Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0P2DX1A2WHX38CS2YA0W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:02.719" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 400.857763] gitlab-workhorse[817]: time="2026-10-04T11:33:02Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 400.859667] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:02 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 61 vm-test-run-nixbot-gitlab> gitlab # [ 400.861769] gitaly[910]: time="2026-10-04T11:33:02.732Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0P2DX1A2WHX38CS2YA0W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:02.719" grpc.time_ms=11.737 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 400.999830] gitaly[910]: time="2026-10-04T11:33:02.898Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-04T11:33:32.782" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:02.882" grpc.time_ms=15.754 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 401.024798] gitaly[910]: time="2026-10-04T11:33:02.915Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M43B0E52BK28RD02GWGHJ25S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-04T11:33:32.802" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:02.902" grpc.time_ms=12.423 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 401.046222] sidekiq-cluster[3304]: {"severity":"WARN","time":"2026-10-04T11:33:02.930Z","class":"CreatePipelineWorker","correlation_id":"01M43B0E52BK28RD02GWGHJ25S","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"baa7927c76a37407367ce093ce7f4d35363a479f","after":"b1c994f1551ecfb35328e5078741a48f638cbb55","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"b1c994f1551ecfb35328e5078741a48f638cbb55","ref":"refs/heads/master"},"retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 401.057602] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:02.936Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"0b57dc3d45922275f80fd1fb","created_at":"2026-10-04T11:32:59.699Z","trace_propagation_headers":{"sentry-trace":"0c83733bdf8a48ca960e4ea6962d1d36-b9e8224d176449db","baggage":"sentry-trace_id=0c83733bdf8a48ca960e4ea6962d1d36,sentry-release=Unknown"},"correlation_id":"01M43B0E52BK28RD02GWGHJ25S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f9c80bd82e479a17b7624aad23d3c004bbd5af740f55bd7a0303df1c2b307d0a","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.702Z","job_size_bytes":467,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-0b57dc3d45922275f80fd1fb: done: 0.84443 sec","job_status":"done","queue_duration_s":2.389782,"scheduling_latency_s":2.389798,"gitaly_calls":7,"gitaly_duration_s":0.065562,"redis_calls":33,"redis_duration_s":0.006849,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000718,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000982,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.00422,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.765905,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.84443,"completed_at":"2026-10-04T11:33:02.936Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016421,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 401.118368] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:02.938Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"581ff242-f26b-45ef-9f5c-d8ce1fff6cc3\"}"],"class":"WebHookWorker","jid":"531bb7364ca98aad950a6309","created_at":"2026-10-04T11:32:59.727Z","trace_propagation_headers":{"sentry-trace":"0c83733bdf8a48ca960e4ea6962d1d36-b9e8224d176449db","baggage":"sentry-trace_id=0c83733bdf8a48ca960e4ea6962d1d36,sentry-release=Unknown"},"correlation_id":"01M43B0E52BK28RD02GWGHJ25S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2397cbe90680eb6cdb384de0478e5005a5debed6e68dee1c31471d34cc33f5b1","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.731Z","job_size_bytes":1749,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-531bb7364ca98aad950a6309: start","job_status":"start","queue_duration_s":3.206275,"scheduling_latency_s":3.206298} vm-test-run-nixbot-gitlab> gitlab # [ 401.137339] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:02.981Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"581ff242-f26b-45ef-9f5c-d8ce1fff6cc3\"}"],"class":"WebHookWorker","jid":"531bb7364ca98aad950a6309","created_at":"2026-10-04T11:32:59.727Z","trace_propagation_headers":{"sentry-trace":"0c83733bdf8a48ca960e4ea6962d1d36-b9e8224d176449db","baggage":"sentry-trace_id=0c83733bdf8a48ca960e4ea6962d1d36,sentry-release=Unknown"},"correlation_id":"01M43B0E52BK28RD02GWGHJ25S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2397cbe90680eb6cdb384de0478e5005a5debed6e68dee1c31471d34cc33f5b1","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.731Z","job_size_bytes":1749,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-531bb7364ca98aad950a6309: done: 0.042991 sec","job_status":"done","queue_duration_s":3.206275,"scheduling_latency_s":3.206298,"redis_calls":14,"redis_duration_s":0.0042840000000000005,"redis_read_bytes":25,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.000363,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003079,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000842,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.006752059999996618,"cpu_s":0.02643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.042991,"completed_at":"2026-10-04T11:33:02.980Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002042,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 401.187195] nixbot[3426]: INFO: ::ffff:127.0.0.1:32956 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted vm-test-run-nixbot-gitlab> gitlab # [ 401.188600] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:02.982Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"163b94d926aef210f7634ac0","created_at":"2026-10-04T11:32:59.873Z","trace_propagation_headers":{"sentry-trace":"c0398371a816422da32c384df5c37679-00fd98929ef44d72","baggage":"sentry-trace_id=c0398371a816422da32c384df5c37679,sentry-release=Unknown"},"correlation_id":"01M43AXZBCPGX6HS1MSC6G2F67","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.878Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-163b94d926aef210f7634ac0: start","job_status":"start","queue_duration_s":3.103765,"scheduling_latency_s":3.103783} vm-test-run-nixbot-gitlab> gitlab # [ 401.205357] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.040Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"163b94d926aef210f7634ac0","created_at":"2026-10-04T11:32:59.873Z","trace_propagation_headers":{"sentry-trace":"c0398371a816422da32c384df5c37679-00fd98929ef44d72","baggage":"sentry-trace_id=c0398371a816422da32c384df5c37679,sentry-release=Unknown"},"correlation_id":"01M43AXZBCPGX6HS1MSC6G2F67","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-04T11:32:59.878Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-163b94d926aef210f7634ac0: done: 0.05776 sec","job_status":"done","queue_duration_s":3.103765,"scheduling_latency_s":3.103783,"redis_calls":19,"redis_duration_s":0.003781,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000813,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000565,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001575,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000828,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00871386,"duration_s":0.05776,"completed_at":"2026-10-04T11:33:03.040Z","load_balancing_strategy":"primary","db_duration_s":0.010488,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 401.252249] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.041Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","56fac43511d059ec041be86b"],"class":"WebHooks::LogExecutionWorker","jid":"455870f0d350bbde63002edc","created_at":"2026-10-04T11:33:01.316Z","trace_propagation_headers":{"sentry-trace":"c0398371a816422da32c384df5c37679-bac7a97e6db84cd6","baggage":"sentry-trace_id=c0398371a816422da32c384df5c37679,sentry-release=Unknown"},"correlation_id":"01M43AXZBCPGX6HS1MSC6G2F67","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:233944c481cf717bd2fafb108f2655a6734d15fa16551bf65db43a8bd639d551","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:01.326Z","job_size_bytes":2592,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-455870f0d350bbde63002edc: start","job_status":"start","queue_duration_s":1.715222,"scheduling_latency_s":1.715241} vm-test-run-nixbot-gitlab> gitlab # [ 401.270789] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.089Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","56fac43511d059ec041be86b"],"class":"WebHooks::LogExecutionWorker","jid":"455870f0d350bbde63002edc","created_at":"2026-10-04T11:33:01.316Z","trace_propagation_headers":{"sentry-trace":"c0398371a816422da32c384df5c37679-bac7a97e6db84cd6","baggage":"sentry-trace_id=c0398371a816422da32c384df5c37679,sentry-release=Unknown"},"correlation_id":"01M43AXZBCPGX6HS1MSC6G2F67","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:233944c481cf717bd2fafb108f2655a6734d15fa16551bf65db43a8bd639d551","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:01.326Z","job_size_bytes":2592,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-455870f0d350bbde63002edc: done: 0.047298 sec","job_status":"done","queue_duration_s":1.715222,"scheduling_latency_s":1.715241,"redis_calls":10,"redis_duration_s":0.002087,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000401,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001249,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000437,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032334,"duration_s":0.047298,"completed_at":"2026-10-04T11:33:03.089Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006996,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 401.317338] gitaly[910]: time="2026-10-04T11:33:03.104Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43B0PCKBAV6XP4PZJ8QQDEQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=200 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:03.095" grpc.time_ms=9.221 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 401.330970] gitaly[910]: time="2026-10-04T11:33:03.167Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M43B0PG3N9KFXYF20X18T42K grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:03.159" pid=910 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 401.341444] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 109 vm-test-run-nixbot-gitlab> gitlab # [ 401.343567] gitlab-workhorse[817]: time="2026-10-04T11:33:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 401.345606] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 61 vm-test-run-nixbot-gitlab> gitlab # [ 401.348473] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.090Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"275c11664701c46fb84c732d","created_at":"2026-10-04T11:33:02.191Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-4ad45b0140aa4dbb","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:02.196Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-275c11664701c46fb84c732d: start","job_status":"start","queue_duration_s":0.893621,"scheduling_latency_s":0.893639} vm-test-run-nixbot-gitlab> gitlab # [ 401.364980] gitaly[910]: time="2026-10-04T11:33:03.168Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0PG3N9KFXYF20X18T42K grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:03.159" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 401.379275] gitaly[910]: time="2026-10-04T11:33:03.168Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0PG3N9KFXYF20X18T42K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:03.159" grpc.time_ms=8.965 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 401.394357] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 58 vm-test-run-nixbot-gitlab> gitlab # [ 401.396494] gitaly[910]: time="2026-10-04T11:33:03.191Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43B0N3271P9HTAFRP4AYZGQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:33:03.190" grpc.time_ms=0.757 pid=910 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 401.406398] gitaly[910]: time="2026-10-04T11:33:03.215Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43B0N3271P9HTAFRP4AYZGQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:33:03.213" grpc.time_ms=1.951 pid=910 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 401.416421] gitaly[910]: time="2026-10-04T11:33:03.241Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43B0N3271P9HTAFRP4AYZGQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:33:03.241" grpc.time_ms=0.367 pid=910 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 401.426284] gitaly[910]: time="2026-10-04T11:33:03.247Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M43B0N3271P9HTAFRP4AYZGQ diskcache=55ac54ee-81b8-48eb-957d-b3d410813007 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T11:33:33.066" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:33:03.166" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 401.436362] gitaly[910]: time="2026-10-04T11:33:03.247Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=74 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7215 command.oublock=0 command.real_time_ms=76 command.spawn_token_fork_ms=0 command.system_time_ms=19 command.user_time_ms=55 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0N3271P9HTAFRP4AYZGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T11:33:33.066" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:33:03.166" grpc.time_ms=80.764 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 401.450441] gitaly[910]: time="2026-10-04T11:33:03.270Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M43B0PKC4747RHBK5NW8N4PR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:03.261" grpc.time_ms=8.656 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 401.465341] gitlab-workhorse[817]: time="2026-10-04T11:33:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 vm-test-run-nixbot-gitlab> gitlab # [ 401.467249] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 67 vm-test-run-nixbot-gitlab> gitlab # [ 401.469341] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.351Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"275c11664701c46fb84c732d","created_at":"2026-10-04T11:33:02.191Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-4ad45b0140aa4dbb","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:02.196Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3304,"job_status":"rescheduled","message":"PipelineProcessWorker JID-275c11664701c46fb84c732d: rescheduled"} vm-test-run-nixbot-gitlab> gitlab # [ 401.485137] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.358Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"275c11664701c46fb84c732d","created_at":"2026-10-04T11:33:02.191Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-4ad45b0140aa4dbb","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:02.196Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-275c11664701c46fb84c732d: done: 0.268433 sec","job_status":"done","queue_duration_s":0.893621,"scheduling_latency_s":0.893639,"gitaly_calls":3,"gitaly_duration_s":0.089588,"redis_calls":67,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.014602,"redis_read_bytes":149,"redis_write_bytes":18098,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000623,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000779,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.002864,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12312,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.004276,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.001088,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001757,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002592,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.008,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.140494,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.2270967,"duration_s":0.268433,"completed_at":"2026-10-04T11:33:03.358Z","load_balancing_strategy":"primary","db_duration_s":0.033707,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 401.538332] gitaly[910]: time="2026-10-04T11:33:03.307Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43B0N3271P9HTAFRP4AYZGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T11:33:33.205" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:03.306" grpc.time_ms=0.99 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 401.551493] gitaly[910]: time="2026-10-04T11:33:03.338Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M43B0PN8NM72A56GDQXZW36S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:03.320" pid=910 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 401.562192] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.360Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","531bb7364ca98aad950a6309"],"class":"WebHooks::LogExecutionWorker","jid":"8a6bebd6bc16a66f2ec4b6f8","created_at":"2026-10-04T11:33:02.969Z","trace_propagation_headers":{"sentry-trace":"0c83733bdf8a48ca960e4ea6962d1d36-68423c3e82744682","baggage":"sentry-trace_id=0c83733bdf8a48ca960e4ea6962d1d36,sentry-release=Unknown"},"correlation_id":"01M43B0E52BK28RD02GWGHJ25S","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ec4ea7bb9369b228caae51273d70b9cc2df786af1ac9611e0591421da9a29e96","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:02.972Z","job_size_bytes":2591,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-8a6bebd6bc16a66f2ec4b6f8: start","job_status":"start","queue_duration_s":0.387991,"scheduling_latency_s":0.388011} vm-test-run-nixbot-gitlab> gitlab # [ 401.579763] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.386Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","531bb7364ca98aad950a6309"],"class":"WebHooks::LogExecutionWorker","jid":"8a6bebd6bc16a66f2ec4b6f8","created_at":"2026-10-04T11:33:02.969Z","trace_propagation_headers":{"sentry-trace":"0c83733bdf8a48ca960e4ea6962d1d36-68423c3e82744682","baggage":"sentry-trace_id=0c83733bdf8a48ca960e4ea6962d1d36,sentry-release=Unknown"},"correlation_id":"01M43B0E52BK28RD02GWGHJ25S","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ec4ea7bb9369b228caae51273d70b9cc2df786af1ac9611e0591421da9a29e96","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:02.972Z","job_size_bytes":2591,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-8a6bebd6bc16a66f2ec4b6f8: done: 0.026381 sec","job_status":"done","queue_duration_s":0.387991,"scheduling_latency_s":0.388011,"redis_calls":7,"redis_duration_s":0.001165,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000844,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000321,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026381,"completed_at":"2026-10-04T11:33:03.386Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002704,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 401.625703] gitaly[910]: time="2026-10-04T11:33:03.339Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1101 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0PN8NM72A56GDQXZW36S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:03.320" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 401.640453] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.388Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"65741262d2253c9b8f34d693","created_at":"2026-10-04T11:33:03.025Z","trace_propagation_headers":{"sentry-trace":"c0398371a816422da32c384df5c37679-76b2c038745c489f","baggage":"sentry-trace_id=c0398371a816422da32c384df5c37679,sentry-release=Unknown"},"correlation_id":"01M43AXZBCPGX6HS1MSC6G2F67","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.031Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-65741262d2253c9b8f34d693: start","job_status":"start","queue_duration_s":0.357332,"scheduling_latency_s":0.357352} vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55' vm-test-run-nixbot-gitlab> gitlab # [ 401.658177] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.506Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"65741262d2253c9b8f34d693","created_at":"2026-10-04T11:33:03.025Z","trace_propagation_headers":{"sentry-trace":"c0398371a816422da32c384df5c37679-76b2c038745c489f","baggage":"sentry-trace_id=c0398371a816422da32c384df5c37679,sentry-release=Unknown"},"correlation_id":"01M43AXZBCPGX6HS1MSC6G2F67","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.031Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-65741262d2253c9b8f34d693: done: 0.118333 sec","job_status":"done","queue_duration_s":0.357332,"scheduling_latency_s":0.357352,"redis_calls":11,"redis_duration_s":0.0030440000000000003,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000324,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002249,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000471,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.091101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032926,"duration_s":0.118333,"completed_at":"2026-10-04T11:33:03.506Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02247,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 401.706828] gitaly[910]: time="2026-10-04T11:33:03.339Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1101 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0PN8NM72A56GDQXZW36S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:03.320" grpc.time_ms=18.913 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 401.721932] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.508Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"1ca57c75a76aa36ddea84197","created_at":"2026-10-04T11:33:03.157Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.160Z","job_size_bytes":23,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-1ca57c75a76aa36ddea84197: start","job_status":"start","queue_duration_s":0.348423,"scheduling_latency_s":0.348444} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55', in 0.07 seconds) vm-test-run-nixbot-gitlab> (finished: subtest: push delivers the webhook, in 84.97 seconds) vm-test-run-nixbot-gitlab> subtest: eval and build post commit statuses vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses vm-test-run-nixbot-gitlab> gitlab # [ 401.739561] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.561Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"1ca57c75a76aa36ddea84197","created_at":"2026-10-04T11:33:03.157Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.160Z","job_size_bytes":23,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-1ca57c75a76aa36ddea84197: done: 0.052562 sec","job_status":"done","queue_duration_s":0.348423,"scheduling_latency_s":0.348444,"redis_calls":7,"redis_duration_s":0.001378,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001002,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000376,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.052562,"completed_at":"2026-10-04T11:33:03.561Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012901,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 401.784932] gitaly[910]: time="2026-10-04T11:33:03.341Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0N3271P9HTAFRP4AYZGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:33.239" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:03.340" grpc.time_ms=0.654 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 401.798152] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:03 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b1c994f1551ecfb35328e5078741a48f638cbb55 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 292 vm-test-run-nixbot-gitlab> gitlab # [ 401.800857] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.562Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c59a92f7a5dad3fe7023df6f","created_at":"2026-10-04T11:33:03.253Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.256Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c59a92f7a5dad3fe7023df6f: start","job_status":"start","queue_duration_s":0.306283,"scheduling_latency_s":0.306302} vm-test-run-nixbot-gitlab> gitlab # [ 401.818301] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.597Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c59a92f7a5dad3fe7023df6f","created_at":"2026-10-04T11:33:03.253Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.256Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c59a92f7a5dad3fe7023df6f: done: 0.034823 sec","job_status":"done","queue_duration_s":0.306283,"scheduling_latency_s":0.306302,"redis_calls":7,"redis_duration_s":0.001205,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000857,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000348,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034823,"completed_at":"2026-10-04T11:33:03.597Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006251,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 401.863184] nixbot[3426]: INFO: ::ffff:127.0.0.1:32968 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b1c994f1551ecfb35328e5078741a48f638cbb55 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab # [ 401.865277] nixbot[3426]: {"ts": "2026-10-04T11:33:03.667Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/b1c994f1551ecfb35328e5078741a48f638cbb55 \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 401.868120] nixbot[3426]: {"ts": "2026-10-04T11:33:03.677Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-b1c994f1551e-5"} vm-test-run-nixbot-gitlab> gitlab # [ 401.870512] gitaly[910]: time="2026-10-04T11:33:03.434Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0PRC2BA4Z5S71NF1VWH2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:33.233" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:03.433" grpc.time_ms=0.531 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 401.885119] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.598Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"d0214842f63e3fb2eadae715","created_at":"2026-10-04T11:33:03.264Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.268Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-d0214842f63e3fb2eadae715: start","job_status":"start","queue_duration_s":0.330479,"scheduling_latency_s":0.330501} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses, in 0.14 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 401.901633] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.635Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"d0214842f63e3fb2eadae715","created_at":"2026-10-04T11:33:03.264Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.268Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-d0214842f63e3fb2eadae715: done: 0.036768 sec","job_status":"done","queue_duration_s":0.330479,"scheduling_latency_s":0.330501,"redis_calls":7,"redis_duration_s":0.003583,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002907,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000676,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036768,"completed_at":"2026-10-04T11:33:03.635Z","load_balancing_strategy":"primary","db_duration_s":0.006202,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 401.945661] gitaly[910]: time="2026-10-04T11:33:03.454Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=589 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43B0PRC2BA4Z5S71NF1VWH2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-04T11:33:33.242" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:33:03.443" grpc.time_ms=11.057 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 401.960045] nixbot[3426]: {"ts": "2026-10-04T11:33:03.807Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-b1c994f1551e-5", "argv0": "bwrap"} vm-test-run-nixbot-gitlab> gitlab # [ 401.962477] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.637Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"06bd3fad2b9cdae912172cb6","created_at":"2026-10-04T11:33:03.278Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.280Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-06bd3fad2b9cdae912172cb6: start","job_status":"start","queue_duration_s":0.357203,"scheduling_latency_s":0.357229} vm-test-run-nixbot-gitlab> gitlab # [ 401.978768] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.681Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"06bd3fad2b9cdae912172cb6","created_at":"2026-10-04T11:33:03.278Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.280Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-06bd3fad2b9cdae912172cb6: done: 0.044004 sec","job_status":"done","queue_duration_s":0.357203,"scheduling_latency_s":0.357229,"redis_calls":10,"redis_duration_s":0.005901,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004276,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000793,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000832,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044004,"completed_at":"2026-10-04T11:33:03.681Z","load_balancing_strategy":"primary","db_duration_s":0.005662,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 402.024871] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:03 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 106 vm-test-run-nixbot-gitlab> gitlab # [ 402.027541] gitaly[910]: time="2026-10-04T11:33:03.469Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M43B0PRC2BA4Z5S71NF1VWH2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T11:33:13.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:33:03.460" grpc.time_ms=8.549 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 402.041644] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.684Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ccd526c4deaf185e801d2cc9","created_at":"2026-10-04T11:33:03.280Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.282Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ccd526c4deaf185e801d2cc9: start","job_status":"start","queue_duration_s":0.401426,"scheduling_latency_s":0.40145} vm-test-run-nixbot-gitlab> gitlab # [ 402.058943] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.717Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ccd526c4deaf185e801d2cc9","created_at":"2026-10-04T11:33:03.280Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.282Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ccd526c4deaf185e801d2cc9: done: 0.032994 sec","job_status":"done","queue_duration_s":0.401426,"scheduling_latency_s":0.40145,"redis_calls":7,"redis_duration_s":0.001268,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000385,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032994,"completed_at":"2026-10-04T11:33:03.717Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004444,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 402.105284] gitaly[910]: time="2026-10-04T11:33:03.489Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M43B0PRC2BA4Z5S71NF1VWH2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-04T11:33:13.038" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:33:03.488" grpc.time_ms=0.278 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 402.116989] nix-daemon[3586]: accepted connection from pid 4627, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 402.118016] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.718Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"314bc41e4c3926008fa5f18f","created_at":"2026-10-04T11:33:03.283Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.285Z","job_size_bytes":19,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-314bc41e4c3926008fa5f18f: start","job_status":"start","queue_duration_s":0.432953,"scheduling_latency_s":0.432978} vm-test-run-nixbot-gitlab> gitlab # [ 402.135252] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.746Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"314bc41e4c3926008fa5f18f","created_at":"2026-10-04T11:33:03.283Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.285Z","job_size_bytes":19,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-314bc41e4c3926008fa5f18f: done: 0.027738 sec","job_status":"done","queue_duration_s":0.432953,"scheduling_latency_s":0.432978,"redis_calls":8,"redis_duration_s":0.001322,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001038,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000284,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027738,"completed_at":"2026-10-04T11:33:03.746Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003842,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 402.179637] gitaly[910]: time="2026-10-04T11:33:03.501Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=568 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0PRC2BA4Z5S71NF1VWH2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-04T11:33:13.043" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:33:03.493" grpc.time_ms=7.58 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 402.193540] gitaly[910]: time="2026-10-04T11:33:03.536Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43B0PRC2BA4Z5S71NF1VWH2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:33:03.535" grpc.time_ms=0.623 pid=910 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 402.203517] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.747Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"fc31d2a3be782aec439a15a1","created_at":"2026-10-04T11:33:03.286Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.288Z","job_size_bytes":66,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-fc31d2a3be782aec439a15a1: start","job_status":"start","queue_duration_s":0.459099,"scheduling_latency_s":0.459123} vm-test-run-nixbot-gitlab> gitlab # [ 402.221745] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.775Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"fc31d2a3be782aec439a15a1","created_at":"2026-10-04T11:33:03.286Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.288Z","job_size_bytes":66,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-fc31d2a3be782aec439a15a1: done: 0.027965 sec","job_status":"done","queue_duration_s":0.459099,"scheduling_latency_s":0.459123,"redis_calls":7,"redis_duration_s":0.0015569999999999998,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001174,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000383,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020631,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027965,"completed_at":"2026-10-04T11:33:03.775Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002759,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 402.267227] gitaly[910]: time="2026-10-04T11:33:03.561Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M43B0PRC2BA4Z5S71NF1VWH2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T11:33:03.561" grpc.time_ms=0.388 pid=910 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 402.277370] gitaly[910]: time="2026-10-04T11:33:03.568Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M43B0PRC2BA4Z5S71NF1VWH2 diskcache=698bb4e1-babc-42c0-b2d4-6659d00f9d17 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T11:33:13.053" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:33:03.503" pid=910 ref=refs/keep-around/b1c994f1551ecfb35328e5078741a48f638cbb55 remote_ip="2001:db8:1::1" revision=b1c994f1551ecfb35328e5078741a48f638cbb55 span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 402.290271] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.776Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"d0f41834d2db95e260c6fef3","created_at":"2026-10-04T11:33:03.288Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.290Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-d0f41834d2db95e260c6fef3: start","job_status":"start","queue_duration_s":0.486145,"scheduling_latency_s":0.486168} vm-test-run-nixbot-gitlab> gitlab # [ 402.307216] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.809Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"d0f41834d2db95e260c6fef3","created_at":"2026-10-04T11:33:03.288Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.290Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-d0f41834d2db95e260c6fef3: done: 0.032259 sec","job_status":"done","queue_duration_s":0.486145,"scheduling_latency_s":0.486168,"redis_calls":7,"redis_duration_s":0.003886,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003537,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000349,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019778,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032259,"completed_at":"2026-10-04T11:33:03.809Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005307,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 402.351952] gitaly[910]: time="2026-10-04T11:33:03.568Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=58 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5030 command.oublock=8 command.real_time_ms=59 command.spawn_token_fork_ms=0 command.system_time_ms=37 command.user_time_ms=21 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0PRC2BA4Z5S71NF1VWH2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T11:33:13.053" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T11:33:03.503" grpc.time_ms=64.807 pid=910 ref=refs/keep-around/b1c994f1551ecfb35328e5078741a48f638cbb55 remote_ip="2001:db8:1::1" revision=b1c994f1551ecfb35328e5078741a48f638cbb55 span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 402.368058] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.810Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"1c3f0387e2a3d604fb03435c","created_at":"2026-10-04T11:33:03.291Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.292Z","job_size_bytes":6,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-1c3f0387e2a3d604fb03435c: start","job_status":"start","queue_duration_s":0.517335,"scheduling_latency_s":0.517359} vm-test-run-nixbot-gitlab> gitlab # [ 402.384417] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.825Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"1c3f0387e2a3d604fb03435c","created_at":"2026-10-04T11:33:03.291Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.292Z","job_size_bytes":6,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-1c3f0387e2a3d604fb03435c: done: 0.014946 sec","job_status":"done","queue_duration_s":0.517335,"scheduling_latency_s":0.517359,"redis_calls":7,"redis_duration_s":0.0010639999999999998,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000783,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000281,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014946,"completed_at":"2026-10-04T11:33:03.825Z","load_balancing_strategy":"primary","db_duration_s":0.000375,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 402.428551] gitaly[910]: time="2026-10-04T11:33:03.572Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0PRC2BA4Z5S71NF1VWH2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:33.071" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:03.571" grpc.time_ms=0.436 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 402.441872] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.826Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"093414dd2d1ef471eb4b5461","created_at":"2026-10-04T11:33:03.293Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.295Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-093414dd2d1ef471eb4b5461: start","job_status":"start","queue_duration_s":0.531304,"scheduling_latency_s":0.531326} vm-test-run-nixbot-gitlab> gitlab # [ 402.457928] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.851Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"093414dd2d1ef471eb4b5461","created_at":"2026-10-04T11:33:03.293Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.295Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-093414dd2d1ef471eb4b5461: done: 0.02546 sec","job_status":"done","queue_duration_s":0.531304,"scheduling_latency_s":0.531326,"redis_calls":7,"redis_duration_s":0.001098,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000798,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0003,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02546,"completed_at":"2026-10-04T11:33:03.851Z","load_balancing_strategy":"primary","db_duration_s":0.003417,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 402.502093] gitaly[910]: time="2026-10-04T11:33:03.763Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0Q2D3QTWEQAPN2X0BBE7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:33.559" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:03.760" grpc.time_ms=3.272 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 402.515519] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.853Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"4241bc34bc4ff7be87be172f","created_at":"2026-10-04T11:33:03.352Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.353Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4241bc34bc4ff7be87be172f: start","job_status":"start","queue_duration_s":0.499211,"scheduling_latency_s":0.499235} vm-test-run-nixbot-gitlab> gitlab # [ 402.531987] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.880Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"4241bc34bc4ff7be87be172f","created_at":"2026-10-04T11:33:03.352Z","trace_propagation_headers":{"sentry-trace":"958fdb1e4ecc434c8f81f4c3b2593a0d-6781d7f9b5b2498b","baggage":"sentry-trace_id=958fdb1e4ecc434c8f81f4c3b2593a0d,sentry-release=Unknown"},"correlation_id":"01M43B0N3271P9HTAFRP4AYZGQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.353Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4241bc34bc4ff7be87be172f: done: 0.027625 sec","job_status":"done","queue_duration_s":0.499211,"scheduling_latency_s":0.499235,"redis_calls":8,"redis_duration_s":0.001417,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001082,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000335,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027625,"completed_at":"2026-10-04T11:33:03.880Z","load_balancing_strategy":"primary","db_duration_s":0.003755,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 402.575900] gitaly[910]: time="2026-10-04T11:33:04.053Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43B0PRC2BA4Z5S71NF1VWH2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T11:33:34.052" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:04.052" grpc.time_ms=0.893 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 402.589261] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.882Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"8a0b49e59714511c1354a775","created_at":"2026-10-04T11:33:03.574Z","trace_propagation_headers":{"sentry-trace":"8efbe014e87440f1a5da5e9438069558-8b231f1ec2f54418","baggage":"sentry-trace_id=8efbe014e87440f1a5da5e9438069558,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43B0PRC2BA4Z5S71NF1VWH2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.578Z","job_size_bytes":65,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-8a0b49e59714511c1354a775: start","job_status":"start","queue_duration_s":0.303243,"scheduling_latency_s":0.303267} vm-test-run-nixbot-gitlab> gitlab # [ 402.606801] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.913Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"8a0b49e59714511c1354a775","created_at":"2026-10-04T11:33:03.574Z","trace_propagation_headers":{"sentry-trace":"8efbe014e87440f1a5da5e9438069558-8b231f1ec2f54418","baggage":"sentry-trace_id=8efbe014e87440f1a5da5e9438069558,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43B0PRC2BA4Z5S71NF1VWH2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.578Z","job_size_bytes":65,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-8a0b49e59714511c1354a775: done: 0.031518 sec","job_status":"done","queue_duration_s":0.303243,"scheduling_latency_s":0.303267,"redis_calls":7,"redis_duration_s":0.0014370000000000001,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001033,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000404,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031518,"completed_at":"2026-10-04T11:33:03.913Z","load_balancing_strategy":"primary","db_duration_s":0.005124,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 402.652395] gitaly[910]: time="2026-10-04T11:33:04.093Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0PRC2BA4Z5S71NF1VWH2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:34.092" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:04.093" grpc.time_ms=0.628 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 402.665233] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.915Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f32d8b4e5ec11027f1729bce","created_at":"2026-10-04T11:33:03.579Z","trace_propagation_headers":{"sentry-trace":"8efbe014e87440f1a5da5e9438069558-8b231f1ec2f54418","baggage":"sentry-trace_id=8efbe014e87440f1a5da5e9438069558,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43B0PRC2BA4Z5S71NF1VWH2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.582Z","job_size_bytes":65,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f32d8b4e5ec11027f1729bce: start","job_status":"start","queue_duration_s":0.332458,"scheduling_latency_s":0.332485} vm-test-run-nixbot-gitlab> gitlab # [ 402.683700] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.974Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f32d8b4e5ec11027f1729bce","created_at":"2026-10-04T11:33:03.579Z","trace_propagation_headers":{"sentry-trace":"8efbe014e87440f1a5da5e9438069558-8b231f1ec2f54418","baggage":"sentry-trace_id=8efbe014e87440f1a5da5e9438069558,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43B0PRC2BA4Z5S71NF1VWH2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.582Z","job_size_bytes":65,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f32d8b4e5ec11027f1729bce: done: 0.059548 sec","job_status":"done","queue_duration_s":0.332458,"scheduling_latency_s":0.332485,"redis_calls":10,"redis_duration_s":0.002839,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001707,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001132,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.059548,"completed_at":"2026-10-04T11:33:03.974Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016405,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 402.730324] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:03.976Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"e015a5fea4f3b86701ebeca4","created_at":"2026-10-04T11:33:03.638Z","trace_propagation_headers":{"sentry-trace":"8efbe014e87440f1a5da5e9438069558-8b231f1ec2f54418","baggage":"sentry-trace_id=8efbe014e87440f1a5da5e9438069558,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43B0PRC2BA4Z5S71NF1VWH2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.647Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e015a5fea4f3b86701ebeca4: start","job_status":"start","queue_duration_s":0.329259,"scheduling_latency_s":0.329286} vm-test-run-nixbot-gitlab> gitlab # [ 402.747017] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:04.105Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"e015a5fea4f3b86701ebeca4","created_at":"2026-10-04T11:33:03.638Z","trace_propagation_headers":{"sentry-trace":"8efbe014e87440f1a5da5e9438069558-8b231f1ec2f54418","baggage":"sentry-trace_id=8efbe014e87440f1a5da5e9438069558,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M43B0PRC2BA4Z5S71NF1VWH2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:03.647Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e015a5fea4f3b86701ebeca4: done: 0.12831 sec","job_status":"done","queue_duration_s":0.329259,"scheduling_latency_s":0.329286,"gitaly_calls":2,"gitaly_duration_s":0.005904,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.006147,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000966,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000913,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000246,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001574,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000236,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000391,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000855,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.09621,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09635212,"duration_s":0.12831,"completed_at":"2026-10-04T11:33:04.104Z","load_balancing_strategy":"primary","db_duration_s":0.02673,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 402.800844] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:04.106Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"9f0627b77db592641e2e2987","created_at":"2026-10-04T11:33:04.036Z","trace_propagation_headers":{"sentry-trace":"8efbe014e87440f1a5da5e9438069558-23e6bf37cdc74781","baggage":"sentry-trace_id=8efbe014e87440f1a5da5e9438069558,sentry-release=Unknown"},"correlation_id":"01M43B0PRC2BA4Z5S71NF1VWH2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:04.038Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-9f0627b77db592641e2e2987: start","job_status":"start","queue_duration_s":0.067694,"scheduling_latency_s":0.067715} vm-test-run-nixbot-gitlab> gitlab # [ 402.817613] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:04.139Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"9f0627b77db592641e2e2987","created_at":"2026-10-04T11:33:04.036Z","trace_propagation_headers":{"sentry-trace":"8efbe014e87440f1a5da5e9438069558-23e6bf37cdc74781","baggage":"sentry-trace_id=8efbe014e87440f1a5da5e9438069558,sentry-release=Unknown"},"correlation_id":"01M43B0PRC2BA4Z5S71NF1VWH2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:04.038Z","job_size_bytes":3,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-9f0627b77db592641e2e2987: done: 0.033622 sec","job_status":"done","queue_duration_s":0.067694,"scheduling_latency_s":0.067715,"redis_calls":7,"redis_duration_s":0.001288,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000392,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033622,"completed_at":"2026-10-04T11:33:04.139Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004767,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses vm-test-run-nixbot-gitlab> gitlab # [ 403.030641] gitaly[910]: time="2026-10-04T11:33:04.929Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0R6QTEC7FYNGEW0P7KDQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:34.427" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:04.928" grpc.time_ms=0.863 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses, in 0.16 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 403.070151] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:04 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 112 vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses vm-test-run-nixbot-gitlab> gitlab # [ 404.167106] gitaly[910]: time="2026-10-04T11:33:06.065Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0SAGYGX2NDZKX16WK4VY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:36.063" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:06.064" grpc.time_ms=0.805 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses, in 0.13 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 404.200114] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:06 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 97 vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses vm-test-run-nixbot-gitlab> gitlab # [ 405.302244] gitaly[910]: time="2026-10-04T11:33:07.200Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0TDTXVX52G3G241V7R4X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:37.098" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:07.199" grpc.time_ms=0.848 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses, in 0.14 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 405.339745] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:07 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 107 vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses vm-test-run-nixbot-gitlab> gitlab # [ 406.353647] nix-daemon[3586]: accepted connection from pid 4658, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 406.368284] nix-daemon[3586]: accepted connection from pid 4627, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 406.438562] gitaly[910]: time="2026-10-04T11:33:08.336Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0VHF71CF8SEH4FE9G8J9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:38.335" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:08.335" grpc.time_ms=0.885 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses, in 0.13 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 406.471961] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:08 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 98 vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses vm-test-run-nixbot-gitlab> gitlab # [ 407.570820] gitaly[910]: time="2026-10-04T11:33:09.469Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0WMVDCSF9W3FR4DD1G2M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:39.467" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:09.467" grpc.time_ms=0.824 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses, in 0.13 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 407.604075] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:09 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 98 vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses vm-test-run-nixbot-gitlab> gitlab # [ 408.706384] gitaly[910]: time="2026-10-04T11:33:10.604Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0XR6338EKSMTF5F0827W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:40.403" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:10.603" grpc.time_ms=0.805 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses, in 0.14 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 408.742945] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:10 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 106 vm-test-run-nixbot-gitlab> gitlab # [ 409.418362] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:11.316Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"87c6b5a257af4ca03554de2a","created_at":"2026-10-04T11:33:11.275Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e75d319100e95a93a9286e4905dcebf","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.314Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-87c6b5a257af4ca03554de2a: start","job_status":"start","queue_duration_s":0.002312,"scheduling_latency_s":0.002346} vm-test-run-nixbot-gitlab> gitlab # [ 409.602111] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:11.500Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"87c6b5a257af4ca03554de2a","created_at":"2026-10-04T11:33:11.275Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e75d319100e95a93a9286e4905dcebf","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.314Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-87c6b5a257af4ca03554de2a: done: 0.184249 sec","job_status":"done","queue_duration_s":0.002312,"scheduling_latency_s":0.002346,"redis_calls":9,"redis_duration_s":0.021824999999999997,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001828,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.018038,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001959,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.184249,"completed_at":"2026-10-04T11:33:11.500Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.057159,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 409.671523] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:11.505Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"0c80c93dec098761258a03bc","created_at":"2026-10-04T11:33:11.329Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.337Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-0c80c93dec098761258a03bc: start","job_status":"start","queue_duration_s":0.168363,"scheduling_latency_s":0.168391} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses vm-test-run-nixbot-gitlab> gitlab # [ 409.748867] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:11.647Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"0c80c93dec098761258a03bc","created_at":"2026-10-04T11:33:11.329Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.337Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-0c80c93dec098761258a03bc: done: 0.141594 sec","job_status":"done","queue_duration_s":0.168363,"scheduling_latency_s":0.168391,"redis_calls":28,"redis_duration_s":0.046722,"redis_read_bytes":36,"redis_write_bytes":10603,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002867,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":680,"redis_queues_calls":2,"redis_queues_duration_s":0.001888,"redis_queues_read_bytes":3,"redis_queues_write_bytes":8315,"redis_queues_metadata_calls":16,"redis_queues_metadata_duration_s":0.034044,"redis_queues_metadata_read_bytes":24,"redis_queues_metadata_write_bytes":1162,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.007923,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":446,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.063003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00094548,"duration_s":0.141594,"completed_at":"2026-10-04T11:33:11.647Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006538,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 409.816923] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:11.668Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"82e3f128685fda131d078d19","created_at":"2026-10-04T11:33:11.363Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bae2bfee861c446f19ec0be2d9fcbd0f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.375Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-82e3f128685fda131d078d19: start","job_status":"start","queue_duration_s":0.292816,"scheduling_latency_s":0.292845} vm-test-run-nixbot-gitlab> gitlab # [ 409.842472] gitaly[910]: time="2026-10-04T11:33:11.740Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0YVXJRJAWTWDNABAA4AK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:41.539" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:11.739" grpc.time_ms=0.687 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses, in 0.13 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 409.875294] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 95 vm-test-run-nixbot-gitlab> gitlab # [ 410.166998] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.065Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"82e3f128685fda131d078d19","created_at":"2026-10-04T11:33:11.363Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bae2bfee861c446f19ec0be2d9fcbd0f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.375Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-82e3f128685fda131d078d19: done: 0.396936 sec","job_status":"done","queue_duration_s":0.292816,"scheduling_latency_s":0.292845,"redis_calls":70,"redis_duration_s":0.090935,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.008927,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.00208,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.07886,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001068,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.190819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.396936,"completed_at":"2026-10-04T11:33:12.065Z","load_balancing_strategy":"primary","db_duration_s":0.037953,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 410.232547] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.067Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"86830b106500e23e247bb01e","created_at":"2026-10-04T11:33:11.391Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b75446044ef6a66bc1a61c2bbb612d07","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T11:33:11.398Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-86830b106500e23e247bb01e: start","job_status":"start","queue_duration_s":0.669137,"scheduling_latency_s":0.669167} vm-test-run-nixbot-gitlab> gitlab # [ 410.248468] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.084Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 410.252250] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.093Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"86830b106500e23e247bb01e","created_at":"2026-10-04T11:33:11.391Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b75446044ef6a66bc1a61c2bbb612d07","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T11:33:11.398Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-86830b106500e23e247bb01e: done: 0.026264 sec","job_status":"done","queue_duration_s":0.669137,"scheduling_latency_s":0.669167,"redis_calls":9,"redis_duration_s":0.001869,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000405,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001058,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000406,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026264,"completed_at":"2026-10-04T11:33:12.093Z","load_balancing_strategy":"primary","db_duration_s":0.00055,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 410.296378] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.095Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"ae282e9c46b1cf1669286e22","created_at":"2026-10-04T11:33:11.411Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7a49e2b4d13b992dc77c06fd76c5a83d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.423Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-ae282e9c46b1cf1669286e22: start","job_status":"start","queue_duration_s":0.671702,"scheduling_latency_s":0.671724} vm-test-run-nixbot-gitlab> gitlab # [ 410.310832] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.134Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"ae282e9c46b1cf1669286e22","created_at":"2026-10-04T11:33:11.411Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7a49e2b4d13b992dc77c06fd76c5a83d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.423Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-ae282e9c46b1cf1669286e22: done: 0.038629 sec","job_status":"done","queue_duration_s":0.671702,"scheduling_latency_s":0.671724,"redis_calls":9,"redis_duration_s":0.001911,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000375,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001126,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00041,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.038629,"completed_at":"2026-10-04T11:33:12.134Z","load_balancing_strategy":"primary","db_duration_s":0.013003,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 410.354822] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.135Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"52b09aea5b71fa1f96d8a7c6","created_at":"2026-10-04T11:33:11.454Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8d055a670bfe4138b30aeab2802d5ea","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.464Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-52b09aea5b71fa1f96d8a7c6: start","job_status":"start","queue_duration_s":0.671449,"scheduling_latency_s":0.671469} vm-test-run-nixbot-gitlab> gitlab # [ 410.369960] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.174Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"52b09aea5b71fa1f96d8a7c6","created_at":"2026-10-04T11:33:11.454Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8d055a670bfe4138b30aeab2802d5ea","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.464Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-52b09aea5b71fa1f96d8a7c6: done: 0.038446 sec","job_status":"done","queue_duration_s":0.671449,"scheduling_latency_s":0.671469,"redis_calls":12,"redis_duration_s":0.0058059999999999995,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000731,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003427,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001648,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024749,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01029048,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.038446,"completed_at":"2026-10-04T11:33:12.174Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003488,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 410.415964] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.176Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"714ee8f90ca57c5208ea0de3","created_at":"2026-10-04T11:33:11.493Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d599c5987d89f30620c6b39dc36e077","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.504Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-714ee8f90ca57c5208ea0de3: start","job_status":"start","queue_duration_s":0.671789,"scheduling_latency_s":0.671816} vm-test-run-nixbot-gitlab> gitlab # [ 410.431727] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.208Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"714ee8f90ca57c5208ea0de3","created_at":"2026-10-04T11:33:11.493Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d599c5987d89f30620c6b39dc36e077","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.504Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-714ee8f90ca57c5208ea0de3: done: 0.032437 sec","job_status":"done","queue_duration_s":0.671789,"scheduling_latency_s":0.671816,"redis_calls":11,"redis_duration_s":0.004058,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000894,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001372,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001792,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020463,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032437,"completed_at":"2026-10-04T11:33:12.208Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00117,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 410.475982] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.210Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"21a622a9e8f59a4182ce114e","created_at":"2026-10-04T11:33:11.509Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e68d113bda614a6e62b826306a56b662","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T11:33:11.524Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-21a622a9e8f59a4182ce114e: start","job_status":"start","queue_duration_s":0.685479,"scheduling_latency_s":0.685502} vm-test-run-nixbot-gitlab> gitlab # [ 410.490968] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.225Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 410.495065] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.234Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"21a622a9e8f59a4182ce114e","created_at":"2026-10-04T11:33:11.509Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e68d113bda614a6e62b826306a56b662","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T11:33:11.524Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-21a622a9e8f59a4182ce114e: done: 0.024397 sec","job_status":"done","queue_duration_s":0.685479,"scheduling_latency_s":0.685502,"redis_calls":9,"redis_duration_s":0.001922,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000417,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001097,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000408,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024397,"completed_at":"2026-10-04T11:33:12.234Z","load_balancing_strategy":"primary","db_duration_s":0.000464,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 410.537903] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.236Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"d671275ed3e706d03d1e1d85","created_at":"2026-10-04T11:33:11.537Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c9879e1c83174cf4433c9cf68ead1771","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.551Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-d671275ed3e706d03d1e1d85: start","job_status":"start","queue_duration_s":0.684792,"scheduling_latency_s":0.684817} vm-test-run-nixbot-gitlab> gitlab # [ 410.553322] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.300Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"d671275ed3e706d03d1e1d85","created_at":"2026-10-04T11:33:11.537Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c9879e1c83174cf4433c9cf68ead1771","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.551Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-d671275ed3e706d03d1e1d85: done: 0.064359 sec","job_status":"done","queue_duration_s":0.684792,"scheduling_latency_s":0.684817,"redis_calls":11,"redis_duration_s":0.002778,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000401,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001177,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.0012,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0007644,"exclusive_lock_hold_duration_s":0.03413246,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.064359,"completed_at":"2026-10-04T11:33:12.300Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016616,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 410.599331] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.302Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e95afe05f51017bad2c3c7ac","created_at":"2026-10-04T11:33:11.579Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f6660b31670ed61cdebc3cf61b3c0e8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T11:33:11.594Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e95afe05f51017bad2c3c7ac: start","job_status":"start","queue_duration_s":0.707458,"scheduling_latency_s":0.707478} vm-test-run-nixbot-gitlab> gitlab # [ 410.613163] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.343Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e95afe05f51017bad2c3c7ac","created_at":"2026-10-04T11:33:11.579Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f6660b31670ed61cdebc3cf61b3c0e8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T11:33:11.594Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e95afe05f51017bad2c3c7ac: done: 0.040933 sec","job_status":"done","queue_duration_s":0.707458,"scheduling_latency_s":0.707478,"redis_calls":9,"redis_duration_s":0.0023049999999999998,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000538,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001298,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000469,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028292,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040933,"completed_at":"2026-10-04T11:33:12.343Z","load_balancing_strategy":"primary","db_duration_s":0.004365,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 410.655895] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.345Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"28125de67a34fcfe90ef063c","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-28125de67a34fcfe90ef063c: start","job_status":"start","queue_duration_s":0.72471,"scheduling_latency_s":0.724734} vm-test-run-nixbot-gitlab> gitlab # [ 410.670161] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"28125de67a34fcfe90ef063c","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-28125de67a34fcfe90ef063c: done: 0.039997 sec","job_status":"done","queue_duration_s":0.72471,"scheduling_latency_s":0.724734,"redis_calls":16,"redis_duration_s":0.003516,"redis_read_bytes":14,"redis_write_bytes":1526,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000455,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":423,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002052,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":802,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040732,"duration_s":0.039997,"completed_at":"2026-10-04T11:33:12.385Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008758,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 410.714109] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"9d40f69d9e06afa17d6fe79c","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-9d40f69d9e06afa17d6fe79c: start","job_status":"start","queue_duration_s":0.76646,"scheduling_latency_s":0.766483} vm-test-run-nixbot-gitlab> gitlab # [ 410.728281] nix-daemon[3586]: accepted connection from pid 4627, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 410.729299] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.408Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"9d40f69d9e06afa17d6fe79c","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-9d40f69d9e06afa17d6fe79c: done: 0.021237 sec","job_status":"done","queue_duration_s":0.76646,"scheduling_latency_s":0.766483,"redis_calls":13,"redis_duration_s":0.002431,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000817,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001614,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01345,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021237,"completed_at":"2026-10-04T11:33:12.408Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001545,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 410.770590] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.409Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"bc28afe377e659225dd9957b","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-bc28afe377e659225dd9957b: start","job_status":"start","queue_duration_s":0.789253,"scheduling_latency_s":0.789274} vm-test-run-nixbot-gitlab> gitlab # [ 410.784898] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"bc28afe377e659225dd9957b","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-bc28afe377e659225dd9957b: done: 0.019878 sec","job_status":"done","queue_duration_s":0.789253,"scheduling_latency_s":0.789274,"redis_calls":13,"redis_duration_s":0.002481,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000783,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001698,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019878,"completed_at":"2026-10-04T11:33:12.429Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001217,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 410.826102] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"22acffb43621fdc4d793fa44","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-22acffb43621fdc4d793fa44: start","job_status":"start","queue_duration_s":0.810655,"scheduling_latency_s":0.810675} vm-test-run-nixbot-gitlab> gitlab # [ 410.839890] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"22acffb43621fdc4d793fa44","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-22acffb43621fdc4d793fa44: done: 0.018854 sec","job_status":"done","queue_duration_s":0.810655,"scheduling_latency_s":0.810675,"redis_calls":13,"redis_duration_s":0.00237,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001579,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018854,"completed_at":"2026-10-04T11:33:12.449Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001144,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses vm-test-run-nixbot-gitlab> gitlab # [ 410.880971] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.451Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"0431cc79a3aebbddd9b00321","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-0431cc79a3aebbddd9b00321: start","job_status":"start","queue_duration_s":0.830953,"scheduling_latency_s":0.830971} vm-test-run-nixbot-gitlab> gitlab # [ 410.893672] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"0431cc79a3aebbddd9b00321","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-0431cc79a3aebbddd9b00321: done: 0.018262 sec","job_status":"done","queue_duration_s":0.830953,"scheduling_latency_s":0.830971,"redis_calls":13,"redis_duration_s":0.0024479999999999997,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000792,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001656,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018262,"completed_at":"2026-10-04T11:33:12.469Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001177,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 410.935625] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"093190a69e7738a34887dc64","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-093190a69e7738a34887dc64: start","job_status":"start","queue_duration_s":0.850555,"scheduling_latency_s":0.850573} vm-test-run-nixbot-gitlab> gitlab # [ 410.950017] gitaly[910]: time="2026-10-04T11:33:12.782Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B0ZVZS193YFPFN4Z960Q4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:42.580" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:12.780" grpc.time_ms=0.978 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 410.963194] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.487Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"093190a69e7738a34887dc64","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-093190a69e7738a34887dc64: done: 0.016862 sec","job_status":"done","queue_duration_s":0.850555,"scheduling_latency_s":0.850573,"redis_calls":13,"redis_duration_s":0.0022329999999999997,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000701,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001532,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010601,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016862,"completed_at":"2026-10-04T11:33:12.487Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001118,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 411.004860] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.489Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"04a839a07af6558b720d7f7d","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-04a839a07af6558b720d7f7d: start","job_status":"start","queue_duration_s":0.868667,"scheduling_latency_s":0.868683} vm-test-run-nixbot-gitlab> gitlab # [ 411.018642] gitaly[910]: time="2026-10-04T11:33:12.832Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=5 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M43B0ZVZS193YFPFN4Z960Q4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T11:33:22.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:33:12.823" grpc.time_ms=8.872 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 411.032526] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.504Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"04a839a07af6558b720d7f7d","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-04a839a07af6558b720d7f7d: done: 0.015949 sec","job_status":"done","queue_duration_s":0.868667,"scheduling_latency_s":0.868683,"redis_calls":13,"redis_duration_s":0.002214,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000721,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001493,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015949,"completed_at":"2026-10-04T11:33:12.504Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000826,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 411.073402] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"b5c6fa1e585b74d2cdd77d89","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-b5c6fa1e585b74d2cdd77d89: start","job_status":"start","queue_duration_s":0.885824,"scheduling_latency_s":0.88584} vm-test-run-nixbot-gitlab> gitlab # [ 411.087340] nixbot[3426]: {"ts": "2026-10-04T11:33:12.922Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/b1c994f1551ecfb35328e5078741a48f638cbb55 \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 411.090708] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:12 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b1c994f1551ecfb35328e5078741a48f638cbb55 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 216 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses, in 0.24 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 411.093446] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.522Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"b5c6fa1e585b74d2cdd77d89","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-b5c6fa1e585b74d2cdd77d89: done: 0.015812 sec","job_status":"done","queue_duration_s":0.885824,"scheduling_latency_s":0.88584,"redis_calls":13,"redis_duration_s":0.00253,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000702,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001828,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009866,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015812,"completed_at":"2026-10-04T11:33:12.521Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000815,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 411.134242] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"8396dc10cd06b340375ab556","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-8396dc10cd06b340375ab556: start","job_status":"start","queue_duration_s":0.902838,"scheduling_latency_s":0.902854} vm-test-run-nixbot-gitlab> gitlab # [ 411.149185] gitaly[910]: time="2026-10-04T11:33:12.970Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B10000FM7S32PHMHR61PF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:42.768" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:12.969" grpc.time_ms=0.867 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 411.162422] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses HTTP/1.1" 200 677 "" "curl/8.21.0" 176 vm-test-run-nixbot-gitlab> gitlab # [ 411.165242] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"8396dc10cd06b340375ab556","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-8396dc10cd06b340375ab556: done: 0.014845 sec","job_status":"done","queue_duration_s":0.902838,"scheduling_latency_s":0.902854,"redis_calls":13,"redis_duration_s":0.001983,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000704,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001279,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009556,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014845,"completed_at":"2026-10-04T11:33:12.538Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000719,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 411.205882] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.539Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"13eb25479cb8a1686f098e10","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-13eb25479cb8a1686f098e10: start","job_status":"start","queue_duration_s":0.918967,"scheduling_latency_s":0.918984} vm-test-run-nixbot-gitlab> gitlab # [ 411.219614] gitaly[910]: time="2026-10-04T11:33:13.010Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B102Y7VVWCYFJ721WCK07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:43.009" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:13.010" grpc.time_ms=0.689 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 411.232938] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"13eb25479cb8a1686f098e10","created_at":"2026-10-04T11:33:11.552Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-c7f2ba72d1834890","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"1a3a3442437cfab796084d57be058b28","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.620Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-13eb25479cb8a1686f098e10: done: 0.017129 sec","job_status":"done","queue_duration_s":0.918967,"scheduling_latency_s":0.918984,"redis_calls":13,"redis_duration_s":0.002268,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001396,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017129,"completed_at":"2026-10-04T11:33:12.556Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00074,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 411.273835] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.558Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"17341fd19e59b1b730506b7c","created_at":"2026-10-04T11:33:11.607Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"032d43412725ba3d343eda266be82941","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.622Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-17341fd19e59b1b730506b7c: start","job_status":"start","queue_duration_s":0.936208,"scheduling_latency_s":0.936232} vm-test-run-nixbot-gitlab> gitlab # [ 411.287900] gitaly[910]: time="2026-10-04T11:33:13.056Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43B102Y7VVWCYFJ721WCK07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T11:33:23.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:33:13.046" grpc.time_ms=10.13 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 411.301869] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:13 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b1c994f1551ecfb35328e5078741a48f638cbb55 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 199 vm-test-run-nixbot-gitlab> gitlab # [ 411.304496] nixbot[3426]: {"ts": "2026-10-04T11:33:13.123Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/b1c994f1551ecfb35328e5078741a48f638cbb55 \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 411.307294] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.609Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"17341fd19e59b1b730506b7c","created_at":"2026-10-04T11:33:11.607Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"032d43412725ba3d343eda266be82941","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.622Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-17341fd19e59b1b730506b7c: done: 0.050823 sec","job_status":"done","queue_duration_s":0.936208,"scheduling_latency_s":0.936232,"redis_calls":18,"redis_duration_s":0.005086,"redis_read_bytes":20,"redis_write_bytes":4702,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001701,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000379,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002207,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000799,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.050823,"completed_at":"2026-10-04T11:33:12.609Z","load_balancing_strategy":"primary","db_duration_s":0.002223,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 411.352494] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.611Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"54918e9386a3eedd68cfafca","created_at":"2026-10-04T11:33:11.640Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8a60b1f9b01ce9f3e6f524a00ddbb62","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.650Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-54918e9386a3eedd68cfafca: start","job_status":"start","queue_duration_s":0.960431,"scheduling_latency_s":0.960461} vm-test-run-nixbot-gitlab> gitlab # [ 411.367243] gitaly[910]: time="2026-10-04T11:33:13.202Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B109KWKXPDKM4BNQKWGJP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T11:33:43.201" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T11:33:13.201" grpc.time_ms=0.656 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 411.380630] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.640Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"54918e9386a3eedd68cfafca","created_at":"2026-10-04T11:33:11.640Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8a60b1f9b01ce9f3e6f524a00ddbb62","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.650Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-54918e9386a3eedd68cfafca: done: 0.029819 sec","job_status":"done","queue_duration_s":0.960431,"scheduling_latency_s":0.960461,"redis_calls":11,"redis_duration_s":0.002445,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0004,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001115,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018546,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00070518,"exclusive_lock_hold_duration_s":0.00376046,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.029819,"completed_at":"2026-10-04T11:33:12.640Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001715,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 411.425303] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.642Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"966acbd64c2958ad8e0b7967","created_at":"2026-10-04T11:33:11.662Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"159065226a2efc8c2bad01c5c0d86469","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.674Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-966acbd64c2958ad8e0b7967: start","job_status":"start","queue_duration_s":0.967668,"scheduling_latency_s":0.96769} vm-test-run-nixbot-gitlab> gitlab # [ 411.439821] gitaly[910]: time="2026-10-04T11:33:13.246Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M43B109KWKXPDKM4BNQKWGJP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T11:33:23.195" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T11:33:13.238" grpc.time_ms=7.842 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 411.454152] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.672Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"966acbd64c2958ad8e0b7967","created_at":"2026-10-04T11:33:11.662Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"159065226a2efc8c2bad01c5c0d86469","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.674Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-966acbd64c2958ad8e0b7967: done: 0.030385 sec","job_status":"done","queue_duration_s":0.967668,"scheduling_latency_s":0.96769,"redis_calls":9,"redis_duration_s":0.001854,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000358,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001096,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0004,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030385,"completed_at":"2026-10-04T11:33:12.672Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002969,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 411.497770] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.674Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a5c705aa93b1edc2bc041b9d","created_at":"2026-10-04T11:33:11.719Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"14f18a875b2078852b429acef090f6e5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.732Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a5c705aa93b1edc2bc041b9d: start","job_status":"start","queue_duration_s":0.941978,"scheduling_latency_s":0.942} vm-test-run-nixbot-gitlab> gitlab # [ 411.512653] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.688Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 411.515894] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.696Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a5c705aa93b1edc2bc041b9d","created_at":"2026-10-04T11:33:11.719Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"14f18a875b2078852b429acef090f6e5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.732Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a5c705aa93b1edc2bc041b9d: done: 0.021912 sec","job_status":"done","queue_duration_s":0.941978,"scheduling_latency_s":0.942,"redis_calls":9,"redis_duration_s":0.002017,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000396,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001141,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00048,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021912,"completed_at":"2026-10-04T11:33:12.696Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000503,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 411.560927] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.697Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"f9afe45c746687ce8be8226c","created_at":"2026-10-04T11:33:11.746Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"19864521564f444b95e3d6d402cb6d5c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.768Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-f9afe45c746687ce8be8226c: start","job_status":"start","queue_duration_s":0.929605,"scheduling_latency_s":0.929625} vm-test-run-nixbot-gitlab> gitlab # [ 411.575364] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.727Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"f9afe45c746687ce8be8226c","created_at":"2026-10-04T11:33:11.746Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"19864521564f444b95e3d6d402cb6d5c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.768Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-f9afe45c746687ce8be8226c: done: 0.029078 sec","job_status":"done","queue_duration_s":0.929605,"scheduling_latency_s":0.929625,"redis_calls":11,"redis_duration_s":0.002844,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00047,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001164,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00121,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017807,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00084696,"exclusive_lock_hold_duration_s":0.00778326,"duration_s":0.029078,"completed_at":"2026-10-04T11:33:12.726Z","load_balancing_strategy":"primary","db_duration_s":0.003591,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 411.619154] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.728Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"c7b05670112a93dc1b441602","created_at":"2026-10-04T11:33:11.786Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"86e8f176e7a53a36fdc2b04c9e6fb13d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.798Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-c7b05670112a93dc1b441602: start","job_status":"start","queue_duration_s":0.929659,"scheduling_latency_s":0.929677} vm-test-run-nixbot-gitlab> gitlab # [ 411.634222] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.740Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 411.638023] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.747Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"c7b05670112a93dc1b441602","created_at":"2026-10-04T11:33:11.786Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"86e8f176e7a53a36fdc2b04c9e6fb13d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.798Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-c7b05670112a93dc1b441602: done: 0.019364 sec","job_status":"done","queue_duration_s":0.929659,"scheduling_latency_s":0.929677,"redis_calls":9,"redis_duration_s":0.0017879999999999999,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000407,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000978,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000403,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011757,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019364,"completed_at":"2026-10-04T11:33:12.747Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0007,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 411.681999] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.749Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"dbfa6abbe8bca61378942500","created_at":"2026-10-04T11:33:11.811Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b90419b7f1d21fd2c1a065b48e1acfd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.826Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-dbfa6abbe8bca61378942500: start","job_status":"start","queue_duration_s":0.922704,"scheduling_latency_s":0.922722} vm-test-run-nixbot-gitlab> gitlab # [ 411.695574] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:13 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b1c994f1551ecfb35328e5078741a48f638cbb55 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 346 vm-test-run-nixbot-gitlab> gitlab # [ 411.698271] nixbot[3426]: {"ts": "2026-10-04T11:33:13.483Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/b1c994f1551ecfb35328e5078741a48f638cbb55 \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 411.701178] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.776Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"dbfa6abbe8bca61378942500","created_at":"2026-10-04T11:33:11.811Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b90419b7f1d21fd2c1a065b48e1acfd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.826Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-dbfa6abbe8bca61378942500: done: 0.027931 sec","job_status":"done","queue_duration_s":0.922704,"scheduling_latency_s":0.922722,"redis_calls":12,"redis_duration_s":0.002391,"redis_read_bytes":11,"redis_write_bytes":1281,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000383,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001001,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":518,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001007,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":476,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00051674,"exclusive_lock_hold_duration_s":0.00968034,"duration_s":0.027931,"completed_at":"2026-10-04T11:33:12.776Z","load_balancing_strategy":"primary","db_duration_s":0.004,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 411.744195] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:12.778Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"d735386104d61e2cef92bc8a","created_at":"2026-10-04T11:33:11.846Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4852fcbb9752880a3933b47dfdddb0a2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.859Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-d735386104d61e2cef92bc8a: start","job_status":"start","queue_duration_s":0.919107,"scheduling_latency_s":0.919125} vm-test-run-nixbot-gitlab> gitlab # [ 411.758643] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:13.193Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"d735386104d61e2cef92bc8a","created_at":"2026-10-04T11:33:11.846Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4852fcbb9752880a3933b47dfdddb0a2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.859Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-d735386104d61e2cef92bc8a: done: 0.41548 sec","job_status":"done","queue_duration_s":0.919107,"scheduling_latency_s":0.919125,"redis_calls":13,"redis_duration_s":0.002465,"redis_read_bytes":11,"redis_write_bytes":1563,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000692,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000885,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.053,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.327761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00053336,"exclusive_lock_hold_duration_s":0.39180514,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.41548,"completed_at":"2026-10-04T11:33:13.193Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.092055,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 411.806580] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:13.195Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"893b370ed44d2d9ee5d9d042","created_at":"2026-10-04T11:33:11.870Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b374cdb74dd960b1728c2d48af581ab","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.882Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-893b370ed44d2d9ee5d9d042: start","job_status":"start","queue_duration_s":1.313121,"scheduling_latency_s":1.313141} vm-test-run-nixbot-gitlab> gitlab # [ 411.821883] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:13.216Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"893b370ed44d2d9ee5d9d042","created_at":"2026-10-04T11:33:11.870Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b374cdb74dd960b1728c2d48af581ab","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.882Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-893b370ed44d2d9ee5d9d042: done: 0.021046 sec","job_status":"done","queue_duration_s":1.313121,"scheduling_latency_s":1.313141,"redis_calls":9,"redis_duration_s":0.0014039999999999999,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000288,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000806,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00031,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021046,"completed_at":"2026-10-04T11:33:13.216Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000466,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 411.866746] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:13.217Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"827fe36402a5f73228c6a282","created_at":"2026-10-04T11:33:11.895Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef64378ec81e2cfa47e8721e36526f15","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.908Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-827fe36402a5f73228c6a282: start","job_status":"start","queue_duration_s":1.309006,"scheduling_latency_s":1.309025} vm-test-run-nixbot-gitlab> gitlab # [ 411.882039] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:13.262Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"827fe36402a5f73228c6a282","created_at":"2026-10-04T11:33:11.895Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef64378ec81e2cfa47e8721e36526f15","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.908Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-827fe36402a5f73228c6a282: done: 0.04484 sec","job_status":"done","queue_duration_s":1.309006,"scheduling_latency_s":1.309025,"redis_calls":11,"redis_duration_s":0.001765,"redis_read_bytes":9,"redis_write_bytes":1509,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000268,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00075,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000747,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025713,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00057864,"exclusive_lock_hold_duration_s":0.02327544,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.04484,"completed_at":"2026-10-04T11:33:13.262Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012172,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 411.929273] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:13.263Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"2848117d457780aaa169cfb7","created_at":"2026-10-04T11:33:11.937Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"139827f23ea646c62258814f22fd7d13","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.950Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-2848117d457780aaa169cfb7: start","job_status":"start","queue_duration_s":1.313235,"scheduling_latency_s":1.313253} vm-test-run-nixbot-gitlab> gitlab # [ 411.944180] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:13.276Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 411.947635] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:13.284Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"2848117d457780aaa169cfb7","created_at":"2026-10-04T11:33:11.937Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"139827f23ea646c62258814f22fd7d13","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:11.950Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-2848117d457780aaa169cfb7: done: 0.020372 sec","job_status":"done","queue_duration_s":1.313235,"scheduling_latency_s":1.313253,"redis_calls":9,"redis_duration_s":0.001379,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000285,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000754,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00034,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020372,"completed_at":"2026-10-04T11:33:13.283Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000494,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 411.993245] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:13.285Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"596105b7173b435037d752ef","created_at":"2026-10-04T11:33:11.974Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9884e41dbba6c9c6e2fae7b3db1045c","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:12.019Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-596105b7173b435037d752ef: start","job_status":"start","queue_duration_s":1.265778,"scheduling_latency_s":1.265795} vm-test-run-nixbot-gitlab> gitlab # [ 412.007646] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:13 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 68 vm-test-run-nixbot-gitlab> gitlab # [ 412.009898] gitlab-workhorse[817]: time="2026-10-04T11:33:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 412.011750] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 71 vm-test-run-nixbot-gitlab> gitlab # [ 412.014217] gitaly[910]: time="2026-10-04T11:33:13.830Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M43B10X3ST6TA8X8F4658JVX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:13.820" grpc.time_ms=10.221 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 412.028306] gitaly[910]: time="2026-10-04T11:33:13.903Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M43B10Z8VE983TEA3AM74ZD6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:13.891" pid=910 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 412.039286] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:13.644Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"596105b7173b435037d752ef","created_at":"2026-10-04T11:33:11.974Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-5aa129f364e647a3","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9884e41dbba6c9c6e2fae7b3db1045c","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T11:33:12.019Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-596105b7173b435037d752ef: done: 0.359406 sec","job_status":"done","queue_duration_s":1.265778,"scheduling_latency_s":1.265795,"redis_calls":564,"redis_duration_s":0.06698899999999999,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000285,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000822,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.065882,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.330345,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.359406,"completed_at":"2026-10-04T11:33:13.644Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000389,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 412.082985] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:13.645Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58ac48b01189cc358e7c1109","created_at":"2026-10-04T11:33:11.739Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-6a2aea3f505c454c","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"bae2bfee861c446f19ec0be2d9fcbd0f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:12.045Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58ac48b01189cc358e7c1109: start","job_status":"start","queue_duration_s":1.600762,"scheduling_latency_s":1.600785} vm-test-run-nixbot-gitlab> gitlab # [ 412.095971] gitaly[910]: time="2026-10-04T11:33:13.903Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B10Z8VE983TEA3AM74ZD6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:13.891" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses vm-test-run-nixbot-gitlab> gitlab # [ 412.110277] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:13.670Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58ac48b01189cc358e7c1109","created_at":"2026-10-04T11:33:11.739Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-6a2aea3f505c454c","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"bae2bfee861c446f19ec0be2d9fcbd0f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:12.045Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58ac48b01189cc358e7c1109: done: 0.024543 sec","job_status":"done","queue_duration_s":1.600762,"scheduling_latency_s":1.600785,"redis_calls":15,"redis_duration_s":0.002508,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000317,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000691,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024543,"completed_at":"2026-10-04T11:33:13.670Z","load_balancing_strategy":"primary","db_duration_s":0.000686,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 412.150827] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:13.671Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dab046d564b61803621c7b7","created_at":"2026-10-04T11:33:11.739Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-6a2aea3f505c454c","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"bae2bfee861c446f19ec0be2d9fcbd0f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:12.045Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dab046d564b61803621c7b7: start","job_status":"start","queue_duration_s":1.626501,"scheduling_latency_s":1.626519} vm-test-run-nixbot-gitlab> gitlab # [ 412.163960] gitaly[910]: time="2026-10-04T11:33:13.904Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M43B10Z8VE983TEA3AM74ZD6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:13.891" grpc.time_ms=12.015 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 412.179758] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:14 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 83 vm-test-run-nixbot-gitlab> gitlab # [ 412.182079] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:13.690Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dab046d564b61803621c7b7","created_at":"2026-10-04T11:33:11.739Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-6a2aea3f505c454c","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"bae2bfee861c446f19ec0be2d9fcbd0f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:12.045Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dab046d564b61803621c7b7: done: 0.018533 sec","job_status":"done","queue_duration_s":1.626501,"scheduling_latency_s":1.626519,"redis_calls":13,"redis_duration_s":0.002323,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001025,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001298,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018533,"completed_at":"2026-10-04T11:33:13.690Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 412.225789] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:13.691Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a90076dd2711ee1159eeb954","created_at":"2026-10-04T11:33:11.739Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-6a2aea3f505c454c","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"bae2bfee861c446f19ec0be2d9fcbd0f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:12.045Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a90076dd2711ee1159eeb954: start","job_status":"start","queue_duration_s":1.646325,"scheduling_latency_s":1.646343} vm-test-run-nixbot-gitlab> gitlab # [ 412.238800] gitaly[910]: time="2026-10-04T11:33:14.023Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M43B112M0CT6W14A5Q57YF4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:14.015" grpc.time_ms=8.076 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses, in 0.14 seconds) vm-test-run-nixbot-gitlab> (finished: subtest: eval and build post commit statuses, in 10.49 seconds) vm-test-run-nixbot-gitlab> (finished: run the VM test script, in 479.20 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 412.252560] gitaly[910]: time="2026-10-04T11:33:14.107Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M43B1159D6WH01PNJ83XC102 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T11:33:14.095" pid=910 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 412.264090] gitlab-workhorse[817]: time="2026-10-04T11:33:14Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 412.266017] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:14 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 83 vm-test-run-nixbot-gitlab> gitlab # [ 412.267946] gitlab-workhorse[817]: gitlab 2001:db8:1::1 - - [2026/10/04:11:33:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/b1c994f1551ecfb35328e5078741a48f638cbb55/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 92 vm-test-run-nixbot-gitlab> test script finished in 479.23s vm-test-run-nixbot-gitlab> cleanup vm-test-run-nixbot-gitlab> kill QemuMachine (pid 46) vm-test-run-nixbot-gitlab> gitlab # [ 412.270996] sidekiq-cluster[3304]: {"severity":"INFO","time":"2026-10-04T11:33:13.707Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a90076dd2711ee1159eeb954","created_at":"2026-10-04T11:33:11.739Z","trace_propagation_headers":{"sentry-trace":"a8aae5fe16bb465ebab56a5c36415a35-6a2aea3f505c454c","baggage":"sentry-trace_id=a8aae5fe16bb465ebab56a5c36415a35,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"bae2bfee861c446f19ec0be2d9fcbd0f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T11:33:12.045Z","job_size_bytes":2,"pid":3304,"sidekiq_tid":"m8hs","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a90076dd2711ee1159eeb954: done: 0.016421 sec","job_status":"done","queue_duration_s":1.646325,"scheduling_latency_s":1.646343,"redis_calls":13,"redis_duration_s":0.0015999999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000632,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14) vm-test-run-nixbot-gitlab> (finished: cleanup, in 1.64 seconds) vm-test-run-nixbot-gitlab> additionally exposed symbols: vm-test-run-nixbot-gitlab> gitlab, vm-test-run-nixbot-gitlab> vlan1, vm-test-run-nixbot-gitlab> start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-10-04T11:31:37.832Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'b1c994f1551ecfb35328e5078741a48f638cbb55', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-10-04T11:33:03.364369Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'b1c994f1551ecfb35328e5078741a48f638cbb55', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T11:33:03.630Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'b1c994f1551ecfb35328e5078741a48f638cbb55', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T11:33:03.630Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'b1c994f1551ecfb35328e5078741a48f638cbb55', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T11:33:03.630Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'b1c994f1551ecfb35328e5078741a48f638cbb55', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T11:33:03.630Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'b1c994f1551ecfb35328e5078741a48f638cbb55', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T11:33:03.630Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'b1c994f1551ecfb35328e5078741a48f638cbb55', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T11:33:03.630Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'b1c994f1551ecfb35328e5078741a48f638cbb55', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T11:33:03.630Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'b1c994f1551ecfb35328e5078741a48f638cbb55', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T11:33:03.630Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'b1c994f1551ecfb35328e5078741a48f638cbb55', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 8s (1 warning)', 'created_at': '2026-10-04T11:33:03.630Z', 'started_at': None, 'finished_at': '2026-10-04T11:33:12.879Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'b1c994f1551ecfb35328e5078741a48f638cbb55', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 8s (1 warning)', 'created_at': '2026-10-04T11:33:03.630Z', 'started_at': None, 'finished_at': '2026-10-04T11:33:12.879Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'b1c994f1551ecfb35328e5078741a48f638cbb55', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-10-04T11:33:13.089Z', 'started_at': None, 'finished_at': '2026-10-04T11:33:13.449Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] post-build step Upload coverage to codecov: ok Skipping codecov: project=Mic92/nixbot attr=checks.aarch64-linux.nixbot-gitlab