these 13 derivations will be built: /nix/store/9v3yhdsrm49v9rawgb097d167a5xi796-python3.13-nixbot.drv /nix/store/jzlh7cnahaw468nh2y95fr5cd0r6vwna-unit-nixbot.service.drv /nix/store/a5nlm04xvyrxg2yha85g6ssi49xd3sx1-system-units.drv /nix/store/nk4fn3cb7aqrqjhdihcvl2d5318iqsyj-etc.drv /nix/store/q49pm0yib6b51r56x4zxvqgdzyisvn6f-activate.drv /nix/store/qa82xyiz7clz9s6kbbxjfzlfzbr55yvb-nixos-system-gitlab-test.drv /nix/store/hk4k784j1j5la40av9j6w3rzhfjv3g79-closure-info.drv /nix/store/w2fv03pzpvfxbg9kg0dcmf91azvflx96-closure-info.drv /nix/store/2jhvvrh6ng0rsw1h9s1qr6wv12li02a8-run-nixos-vm.drv /nix/store/p7dd2bp8anj6n7238hik6idrji4hrlf8-nixos-vm.drv /nix/store/k78kcvv40pnsab6qy4l9zagj7ygrlizh-driverConfiguration.json.drv /nix/store/2yqdqlrwvfjpvyk3njxcvajpp10cc68y-nixos-test-driver-nixbot-gitlab.drv /nix/store/iymqxxfbf7dplj6nckmmciasz0mahz1g-vm-test-run-nixbot-gitlab.drv Failed to find a machine for remote build! derivation: 9v3yhdsrm49v9rawgb097d167a5xi796-python3.13-nixbot.drv required (system, features): (aarch64-linux, []) 1 available machines: (systems, maxjobs, supportedFeatures, mandatoryFeatures) ([aarch64-darwin, x86_64-darwin], 8, [big-parallel, recursive-nix], []) building '/nix/store/9v3yhdsrm49v9rawgb097d167a5xi796-python3.13-nixbot.drv' python3.13-nixbot> tribuchet: building on eliza python3.13-nixbot> Sourcing python-remove-tests-dir-hook python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh python3.13-nixbot> Sourcing python-remove-bin-bytecode-hook.sh python3.13-nixbot> Sourcing pypa-build-hook python3.13-nixbot> Using pypaBuildPhase python3.13-nixbot> Sourcing python-runtime-deps-check-hook python3.13-nixbot> Using pythonRuntimeDepsCheckHook python3.13-nixbot> Sourcing pypa-install-hook python3.13-nixbot> Using pypaInstallPhase python3.13-nixbot> Sourcing python-imports-check-hook.sh python3.13-nixbot> Using pythonImportsCheckPhase python3.13-nixbot> Sourcing python-namespaces-hook python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh python3.13-nixbot> Running phase: unpackPhase python3.13-nixbot> unpacking source archive /nix/store/gd13lbybiavrhmxs3j4xsb7j7dmvj1az-nixbot python3.13-nixbot> source root is nixbot python3.13-nixbot> setting SOURCE_DATE_EPOCH to timestamp 315619200 of file "nixbot/pyproject.toml" python3.13-nixbot> Running phase: patchPhase python3.13-nixbot> Running phase: updateAutotoolsGnuConfigScriptsPhase python3.13-nixbot> Running phase: configurePhase python3.13-nixbot> no configure script, doing nothing python3.13-nixbot> Running phase: buildPhase python3.13-nixbot> Executing pypaBuildPhase python3.13-nixbot> Creating a wheel... python3.13-nixbot> pypa build flags: --no-isolation --outdir dist/ --wheel python3.13-nixbot> * Getting build dependencies for wheel... python3.13-nixbot> * Building wheel... python3.13-nixbot> Successfully built nixbot-0.0.1-py3-none-any.whl python3.13-nixbot> Finished creating a wheel... python3.13-nixbot> Finished executing pypaBuildPhase python3.13-nixbot> Running phase: pythonRuntimeDepsCheckHook python3.13-nixbot> Executing pythonRuntimeDepsCheck python3.13-nixbot> Checking runtime dependencies for nixbot-0.0.1-py3-none-any.whl python3.13-nixbot> Finished executing pythonRuntimeDepsCheck python3.13-nixbot> Running phase: installPhase python3.13-nixbot> Executing pypaInstallPhase python3.13-nixbot> Successfully installed nixbot-0.0.1-py3-none-any.whl python3.13-nixbot> Finished executing pypaInstallPhase python3.13-nixbot> Running phase: pythonOutputDistPhase python3.13-nixbot> Executing pythonOutputDistPhase python3.13-nixbot> Finished executing pythonOutputDistPhase python3.13-nixbot> Running phase: fixupPhase python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/b39hz681s4gx4sz11blq6id8zwmqqy7v-python3.13-nixbot python3.13-nixbot> checking for references to /build/ in /nix/store/b39hz681s4gx4sz11blq6id8zwmqqy7v-python3.13-nixbot... python3.13-nixbot> patching script interpreter paths in /nix/store/b39hz681s4gx4sz11blq6id8zwmqqy7v-python3.13-nixbot python3.13-nixbot> stripping (with command strip and flags -S -p) in /nix/store/b39hz681s4gx4sz11blq6id8zwmqqy7v-python3.13-nixbot/lib /nix/store/b39hz681s4gx4sz11blq6id8zwmqqy7v-python3.13-nixbot/bin python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/vgm6ldmgw2mm32a8rjhkfdidwyk8pgd5-python3.13-nixbot-dist python3.13-nixbot> checking for references to /build/ in /nix/store/vgm6ldmgw2mm32a8rjhkfdidwyk8pgd5-python3.13-nixbot-dist... python3.13-nixbot> patching script interpreter paths in /nix/store/vgm6ldmgw2mm32a8rjhkfdidwyk8pgd5-python3.13-nixbot-dist python3.13-nixbot> Rewriting #!/nix/store/lqn6mbgzzdrqq2qkwddcmxj9z6amdd86-python3-3.13.13/bin/python3.13 to #!/nix/store/lqn6mbgzzdrqq2qkwddcmxj9z6amdd86-python3-3.13.13 python3.13-nixbot> wrapping `/nix/store/b39hz681s4gx4sz11blq6id8zwmqqy7v-python3.13-nixbot/bin/nixbot'... python3.13-nixbot> Executing pythonRemoveTestsDir python3.13-nixbot> Finished executing pythonRemoveTestsDir python3.13-nixbot> Running phase: pythonCatchConflictsPhase python3.13-nixbot> Running phase: pythonRemoveBinBytecodePhase python3.13-nixbot> Running phase: pythonImportsCheckPhase python3.13-nixbot> Executing pythonImportsCheckPhase building '/nix/store/jzlh7cnahaw468nh2y95fr5cd0r6vwna-unit-nixbot.service.drv' unit-nixbot.service> tribuchet: building on eliza building '/nix/store/a5nlm04xvyrxg2yha85g6ssi49xd3sx1-system-units.drv' system-units> tribuchet: building on eliza building '/nix/store/nk4fn3cb7aqrqjhdihcvl2d5318iqsyj-etc.drv' etc> tribuchet: building on eliza building '/nix/store/q49pm0yib6b51r56x4zxvqgdzyisvn6f-activate.drv' activate> tribuchet: building on eliza building '/nix/store/qa82xyiz7clz9s6kbbxjfzlfzbr55yvb-nixos-system-gitlab-test.drv' nixos-system-gitlab-test> tribuchet: building on eliza nixos-system-gitlab-test> structuredAttrs is enabled building '/nix/store/hk4k784j1j5la40av9j6w3rzhfjv3g79-closure-info.drv' closure-info> tribuchet: building on eliza closure-info> structuredAttrs is enabled building '/nix/store/w2fv03pzpvfxbg9kg0dcmf91azvflx96-closure-info.drv' closure-info> tribuchet: building on eliza closure-info> structuredAttrs is enabled building '/nix/store/2jhvvrh6ng0rsw1h9s1qr6wv12li02a8-run-nixos-vm.drv' run-nixos-vm> tribuchet: building on eliza building '/nix/store/p7dd2bp8anj6n7238hik6idrji4hrlf8-nixos-vm.drv' nixos-vm> tribuchet: building on eliza building '/nix/store/k78kcvv40pnsab6qy4l9zagj7ygrlizh-driverConfiguration.json.drv' driverConfiguration.json> tribuchet: building on eliza driverConfiguration.json> structuredAttrs is enabled building '/nix/store/2yqdqlrwvfjpvyk3njxcvajpp10cc68y-nixos-test-driver-nixbot-gitlab.drv' nixos-test-driver-nixbot-gitlab> tribuchet: building on eliza nixos-test-driver-nixbot-gitlab> Running type check (enable/disable: config.skipTypeCheck) nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck nixos-test-driver-nixbot-gitlab> All checks passed! nixos-test-driver-nixbot-gitlab> Linting test script (enable/disable: config.skipLint) nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint nixos-test-driver-nixbot-gitlab> All checks passed! Failed to find a machine for remote build! derivation: iymqxxfbf7dplj6nckmmciasz0mahz1g-vm-test-run-nixbot-gitlab.drv required (system, features): (aarch64-linux, [kvm, nixos-test]) 1 available machines: (systems, maxjobs, supportedFeatures, mandatoryFeatures) ([aarch64-darwin, x86_64-darwin], 8, [big-parallel, recursive-nix], []) building '/nix/store/iymqxxfbf7dplj6nckmmciasz0mahz1g-vm-test-run-nixbot-gitlab.drv' vm-test-run-nixbot-gitlab> tribuchet: building on eliza vm-test-run-nixbot-gitlab> Machine state will be reset. To keep it, pass --keep-machine-state vm-test-run-nixbot-gitlab> start all VLans vm-test-run-nixbot-gitlab> (finished: start all VLans, in 0.00 seconds) vm-test-run-nixbot-gitlab> Test will time out and terminate in 3600 seconds vm-test-run-nixbot-gitlab> run the VM test script vm-test-run-nixbot-gitlab> start all VMs vm-test-run-nixbot-gitlab> gitlab: starting vm vm-test-run-nixbot-gitlab> gitlab: QEMU running (pid 13) vm-test-run-nixbot-gitlab> gitlab # Disk image does not exist, creating the virtualisation disk image... vm-test-run-nixbot-gitlab> gitlab # Formatting '/build/vm-state-gitlab/tmp.sOdnkMsHaQ', fmt=raw size=1073741824 vm-test-run-nixbot-gitlab> (finished: start all VMs, in 68.70 seconds) vm-test-run-nixbot-gitlab> gitlab # mke2fs 1.47.3 (8-Jul-2025) vm-test-run-nixbot-gitlab> gitlab: waiting for unit setup-gitlab.service vm-test-run-nixbot-gitlab> gitlab # Discarding device blocks: 0/262144 done vm-test-run-nixbot-gitlab> gitlab: waiting for the VM to finish booting vm-test-run-nixbot-gitlab> gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes vm-test-run-nixbot-gitlab> gitlab # Filesystem UUID: 3a2117c2-08df-4e8c-b828-3fdf80aec035 vm-test-run-nixbot-gitlab> gitlab # Superblock backups stored on blocks: vm-test-run-nixbot-gitlab> gitlab # 32768, 98304, 163840, 229376 vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # Allocating group tables: 0/8 done vm-test-run-nixbot-gitlab> gitlab # Writing inode tables: 0/8 done vm-test-run-nixbot-gitlab> gitlab # Creating journal (8192 blocks): done vm-test-run-nixbot-gitlab> gitlab # Writing superblocks and filesystem accounting information: 0/8 done vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # Virtualisation disk image created. vm-test-run-nixbot-gitlab> gitlab # Creating Nix store image... vm-test-run-nixbot-gitlab> gitlab # Created Nix store image. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Jun 1 15:51:08 UTC 2026 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] KASLR enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] random: crng init done vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Machine model: linux,dummy-virt vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] efi: UEFI not found. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Zone ranges: vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMA32 empty vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Device empty vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Movable zone start for each node vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Early memory node ranges vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: probing for conduit method from DT. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: Trusted OS migration not required vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: SMC Calling Convention v1.1 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s185944 r8192 d117160 u311296 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Detected PIPT I-cache on CPU0 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: Spectre-v4 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: Spectre-BHB vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] alternatives: applying boot alternatives vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rc187vq3l99p7w3vg2rwxlx21fkmlgvh-nixos-system-gitlab-test/init regInfo=/nix/store/g814sg8w0zzncvp8lvhg133pg0mm1wqb-closure-info/registration console=ttyAMA0,115200n8 console=tty0 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/g814sg8w0zzncvp8lvhg133pg0mm1wqb-closure-info/registration", will be passed to user space. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] software IO TLB: area num 4. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Fallback order for Node 0: 0 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1572864 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Policy zone: Normal vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] allocated 12582912 bytes of page_ext vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ftrace: allocating 74648 entries in 292 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ftrace: allocated 292 pages with 3 groups vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: Hierarchical RCU implementation. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: RCU event tracing is enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Rude variant of Tasks RCU enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: 256 SPIs implemented vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f00000 (indirect, esz 8, psz 64K, shr 1) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f10000 (flat, esz 8, psz 64K, shr 1) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f20000 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f30000 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns vm-test-run-nixbot-gitlab> gitlab # [ 0.000030] arm-pv: using stolen time PV vm-test-run-nixbot-gitlab> gitlab # [ 0.000406] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) vm-test-run-nixbot-gitlab> gitlab # [ 0.000585] Console: colour dummy device 80x25 vm-test-run-nixbot-gitlab> gitlab # [ 0.000592] printk: legacy console [tty0] enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.000771] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) vm-test-run-nixbot-gitlab> gitlab # [ 0.000778] pid_max: default: 32768 minimum: 301 vm-test-run-nixbot-gitlab> gitlab # [ 0.000869] LSM: initializing lsm=capability,landlock,yama,bpf,ima vm-test-run-nixbot-gitlab> gitlab # [ 0.001022] landlock: Up and running. vm-test-run-nixbot-gitlab> gitlab # [ 0.001025] Yama: becoming mindful. vm-test-run-nixbot-gitlab> gitlab # [ 0.001473] LSM support for eBPF active vm-test-run-nixbot-gitlab> gitlab # [ 0.001684] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.001796] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.003003] cacheinfo: Unable to detect cache hierarchy for CPU 0 vm-test-run-nixbot-gitlab> gitlab # [ 0.003740] rcu: Hierarchical SRCU implementation. vm-test-run-nixbot-gitlab> gitlab # [ 0.003744] rcu: Max phase no-delay instances is 1000. vm-test-run-nixbot-gitlab> gitlab # [ 0.003905] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level vm-test-run-nixbot-gitlab> gitlab # [ 0.004975] fsl-mc MSI: its@8080000 domain created vm-test-run-nixbot-gitlab> gitlab # [ 0.005069] EFI services will not be available. vm-test-run-nixbot-gitlab> gitlab # [ 0.005249] smp: Bringing up secondary CPUs ... vm-test-run-nixbot-gitlab> gitlab # [ 0.005935] Detected PIPT I-cache on CPU1 vm-test-run-nixbot-gitlab> gitlab # [ 0.006034] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000 vm-test-run-nixbot-gitlab> gitlab # [ 0.006168] GICv3: CPU1: using allocated LPI pending table @0x0000000100f40000 vm-test-run-nixbot-gitlab> gitlab # [ 0.006300] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40] vm-test-run-nixbot-gitlab> gitlab # [ 0.007452] Detected PIPT I-cache on CPU2 vm-test-run-nixbot-gitlab> gitlab # [ 0.007640] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000 vm-test-run-nixbot-gitlab> gitlab # [ 0.007772] GICv3: CPU2: using allocated LPI pending table @0x0000000100f50000 vm-test-run-nixbot-gitlab> gitlab # [ 0.007905] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40] vm-test-run-nixbot-gitlab> gitlab # [ 0.008971] Detected PIPT I-cache on CPU3 vm-test-run-nixbot-gitlab> gitlab # [ 0.009091] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000 vm-test-run-nixbot-gitlab> gitlab # [ 0.009222] GICv3: CPU3: using allocated LPI pending table @0x0000000100f60000 vm-test-run-nixbot-gitlab> gitlab # [ 0.009354] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40] vm-test-run-nixbot-gitlab> gitlab # [ 0.009913] smp: Brought up 1 node, 4 CPUs vm-test-run-nixbot-gitlab> gitlab # [ 0.009940] SMP: Total of 4 processors activated. vm-test-run-nixbot-gitlab> gitlab # [ 0.009942] CPU: All CPU(s) started at EL1 vm-test-run-nixbot-gitlab> gitlab # [ 0.009952] CPU features: detected: Branch Target Identification vm-test-run-nixbot-gitlab> gitlab # [ 0.009956] CPU features: detected: ARMv8.4 Translation Table Level vm-test-run-nixbot-gitlab> gitlab # [ 0.009959] CPU features: detected: Instruction cache invalidation not required for I/D coherence vm-test-run-nixbot-gitlab> gitlab # [ 0.009962] CPU features: detected: Data cache clean to the PoU not required for I/D coherence vm-test-run-nixbot-gitlab> gitlab # [ 0.009966] CPU features: detected: Common not Private translations vm-test-run-nixbot-gitlab> gitlab # [ 0.009969] CPU features: detected: CRC32 instructions vm-test-run-nixbot-gitlab> gitlab # [ 0.009971] CPU features: detected: Data cache clean to Point of Deep Persistence vm-test-run-nixbot-gitlab> gitlab # [ 0.009975] CPU features: detected: Data cache clean to Point of Persistence vm-test-run-nixbot-gitlab> gitlab # [ 0.009977] CPU features: detected: Data independent timing control (DIT) vm-test-run-nixbot-gitlab> gitlab # [ 0.009980] CPU features: detected: E0PD vm-test-run-nixbot-gitlab> gitlab # [ 0.009983] CPU features: detected: Enhanced Counter Virtualization vm-test-run-nixbot-gitlab> gitlab # [ 0.009986] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) vm-test-run-nixbot-gitlab> gitlab # [ 0.009989] CPU features: detected: Enhanced Virtualization Traps vm-test-run-nixbot-gitlab> gitlab # [ 0.009992] CPU features: detected: Fine Grained Traps vm-test-run-nixbot-gitlab> gitlab # [ 0.009994] CPU features: detected: Generic authentication (architected QARMA5 algorithm) vm-test-run-nixbot-gitlab> gitlab # [ 0.009999] CPU features: detected: RCpc load-acquire (LDAPR) vm-test-run-nixbot-gitlab> gitlab # [ 0.010002] CPU features: detected: LSE atomic instructions vm-test-run-nixbot-gitlab> gitlab # [ 0.010004] CPU features: detected: Privileged Access Never vm-test-run-nixbot-gitlab> gitlab # [ 0.010007] CPU features: detected: PMUv3 vm-test-run-nixbot-gitlab> gitlab # [ 0.010010] CPU features: detected: RAS Extension Support vm-test-run-nixbot-gitlab> gitlab # [ 0.010012] CPU features: detected: RASv1p1 Extension Support vm-test-run-nixbot-gitlab> gitlab # [ 0.010015] CPU features: detected: Random Number Generator vm-test-run-nixbot-gitlab> gitlab # [ 0.010017] CPU features: detected: Speculation barrier (SB) vm-test-run-nixbot-gitlab> gitlab # [ 0.010020] CPU features: detected: Stage-2 Force Write-Back vm-test-run-nixbot-gitlab> gitlab # [ 0.010023] CPU features: detected: TLB range maintenance instructions vm-test-run-nixbot-gitlab> gitlab # [ 0.010027] CPU features: detected: Speculative Store Bypassing Safe (SSBS) vm-test-run-nixbot-gitlab> gitlab # [ 0.010155] alternatives: applying system-wide alternatives vm-test-run-nixbot-gitlab> gitlab # [ 0.013166] CPU features: detected: BBM Level 2 without TLB conflict abort vm-test-run-nixbot-gitlab> gitlab # [ 0.013407] Memory: 5965832K/6291456K available (24320K kernel code, 7086K rwdata, 26308K rodata, 4736K init, 1102K bss, 270904K reserved, 32768K cma-reserved) vm-test-run-nixbot-gitlab> gitlab # [ 0.014945] devtmpfs: initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.018879] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.018926] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.018992] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). vm-test-run-nixbot-gitlab> gitlab # [ 0.019203] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL vm-test-run-nixbot-gitlab> gitlab # [ 0.019207] 0 pages in range for non-PLT usage vm-test-run-nixbot-gitlab> gitlab # [ 0.019209] 508336 pages in range for PLT usage vm-test-run-nixbot-gitlab> gitlab # [ 0.019329] pinctrl core: initialized pinctrl subsystem vm-test-run-nixbot-gitlab> gitlab # [ 0.020179] DMI not present or invalid. vm-test-run-nixbot-gitlab> gitlab # [ 0.023590] NET: Registered PF_NETLINK/PF_ROUTE protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.026249] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations vm-test-run-nixbot-gitlab> gitlab # [ 0.026536] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations vm-test-run-nixbot-gitlab> gitlab # [ 0.028397] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations vm-test-run-nixbot-gitlab> gitlab # [ 0.028421] audit: initializing netlink subsys (disabled) vm-test-run-nixbot-gitlab> gitlab # [ 0.028741] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1 vm-test-run-nixbot-gitlab> gitlab # [ 0.029314] thermal_sys: Registered thermal governor 'fair_share' vm-test-run-nixbot-gitlab> gitlab # [ 0.029317] thermal_sys: Registered thermal governor 'bang_bang' vm-test-run-nixbot-gitlab> gitlab # [ 0.029321] thermal_sys: Registered thermal governor 'step_wise' vm-test-run-nixbot-gitlab> gitlab # [ 0.029324] thermal_sys: Registered thermal governor 'user_space' vm-test-run-nixbot-gitlab> gitlab # [ 0.029326] thermal_sys: Registered thermal governor 'power_allocator' vm-test-run-nixbot-gitlab> gitlab # [ 0.029403] cpuidle: using governor ladder vm-test-run-nixbot-gitlab> gitlab # [ 0.029427] cpuidle: using governor menu vm-test-run-nixbot-gitlab> gitlab # [ 0.029642] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. vm-test-run-nixbot-gitlab> gitlab # [ 0.029854] ASID allocator initialised with 65536 entries vm-test-run-nixbot-gitlab> gitlab # [ 0.030988] Serial: AMBA PL011 UART driver vm-test-run-nixbot-gitlab> gitlab # [ 0.036130] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 vm-test-run-nixbot-gitlab> gitlab # [ 0.036444] printk: console [ttyAMA0] enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.048012] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.048031] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page vm-test-run-nixbot-gitlab> gitlab # [ 0.048041] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.048049] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page vm-test-run-nixbot-gitlab> gitlab # [ 0.048057] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.048065] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page vm-test-run-nixbot-gitlab> gitlab # [ 0.048073] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.048081] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page vm-test-run-nixbot-gitlab> gitlab # [ 0.060839] fbcon: Taking over console vm-test-run-nixbot-gitlab> gitlab # [ 0.060852] ACPI: Interpreter disabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.061902] iommu: Default domain type: Translated vm-test-run-nixbot-gitlab> gitlab # [ 0.061918] iommu: DMA domain TLB invalidation policy: strict mode vm-test-run-nixbot-gitlab> gitlab # [ 0.076523] SCSI subsystem initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.077405] usbcore: registered new interface driver usbfs vm-test-run-nixbot-gitlab> gitlab # [ 0.077520] usbcore: registered new interface driver hub vm-test-run-nixbot-gitlab> gitlab # [ 0.077567] usbcore: registered new device driver usb vm-test-run-nixbot-gitlab> gitlab # [ 0.078510] pps_core: LinuxPPS API ver. 1 registered vm-test-run-nixbot-gitlab> gitlab # [ 0.078522] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti vm-test-run-nixbot-gitlab> gitlab # [ 0.078544] PTP clock support registered vm-test-run-nixbot-gitlab> gitlab # [ 0.078645] EDAC MC: Ver: 3.0.0 vm-test-run-nixbot-gitlab> gitlab # [ 0.079464] scmi_core: SCMI protocol bus registered vm-test-run-nixbot-gitlab> gitlab # [ 0.080676] FPGA manager framework vm-test-run-nixbot-gitlab> gitlab # [ 0.083400] vgaarb: loaded vm-test-run-nixbot-gitlab> gitlab # [ 0.085036] clocksource: Switched to clocksource arch_sys_counter vm-test-run-nixbot-gitlab> gitlab # [ 0.086600] VFS: Disk quotas dquot_6.6.0 vm-test-run-nixbot-gitlab> gitlab # [ 0.086663] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) vm-test-run-nixbot-gitlab> gitlab # [ 0.089745] netfs: FS-Cache loaded vm-test-run-nixbot-gitlab> gitlab # [ 0.090001] pnp: PnP ACPI: disabled vm-test-run-nixbot-gitlab> gitlab # [ 0.103544] NET: Registered PF_INET protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.105948] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.159648] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.159727] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.159792] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.160182] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.160989] TCP: Hash tables configured (established 65536 bind 65536) vm-test-run-nixbot-gitlab> gitlab # [ 0.161200] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.161287] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.161462] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.161735] NET: Registered PF_UNIX/PF_LOCAL protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.161780] NET: Registered PF_XDP protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.161801] PCI: CLS 0 bytes, default 64 vm-test-run-nixbot-gitlab> gitlab # [ 0.162114] Trying to unpack rootfs image as initramfs... vm-test-run-nixbot-gitlab> gitlab # [ 0.177159] kvm [1]: HYP mode not available vm-test-run-nixbot-gitlab> gitlab # [ 0.246372] Initialise system trusted keyrings vm-test-run-nixbot-gitlab> gitlab # [ 0.246878] workingset: timestamp_bits=42 max_order=21 bucket_order=0 vm-test-run-nixbot-gitlab> gitlab # [ 0.249497] squashfs: version 4.0 (2009/01/31) Phillip Lougher vm-test-run-nixbot-gitlab> gitlab # [ 0.249702] 9p: Installing v9fs 9p2000 file system support vm-test-run-nixbot-gitlab> gitlab # [ 0.288501] Key type asymmetric registered vm-test-run-nixbot-gitlab> gitlab # [ 0.288531] Asymmetric key parser 'x509' registered vm-test-run-nixbot-gitlab> gitlab # [ 0.288804] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) vm-test-run-nixbot-gitlab> gitlab # [ 0.289226] io scheduler mq-deadline registered vm-test-run-nixbot-gitlab> gitlab # [ 0.289238] io scheduler kyber registered vm-test-run-nixbot-gitlab> gitlab # [ 0.309254] pl061_gpio 9030000.pl061: PL061 GPIO chip registered vm-test-run-nixbot-gitlab> gitlab # [ 0.313123] ledtrig-cpu: registered to indicate activity on CPUs vm-test-run-nixbot-gitlab> gitlab # [ 0.314309] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: vm-test-run-nixbot-gitlab> gitlab # [ 0.314345] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.314395] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.314415] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.314460] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits vm-test-run-nixbot-gitlab> gitlab # [ 0.314512] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] vm-test-run-nixbot-gitlab> gitlab # [ 0.314713] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 vm-test-run-nixbot-gitlab> gitlab # [ 0.314731] pci_bus 0000:00: root bus resource [bus 00-ff] vm-test-run-nixbot-gitlab> gitlab # [ 0.314741] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.314750] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.314759] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.314990] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.316475] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.317158] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] vm-test-run-nixbot-gitlab> gitlab # [ 0.317196] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.317279] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.317317] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.318976] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.319600] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] vm-test-run-nixbot-gitlab> gitlab # [ 0.319636] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.319717] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.321327] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.322006] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f] vm-test-run-nixbot-gitlab> gitlab # [ 0.322042] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.322123] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.323700] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.324330] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] vm-test-run-nixbot-gitlab> gitlab # [ 0.324366] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.324447] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.326215] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.326863] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f] vm-test-run-nixbot-gitlab> gitlab # [ 0.326899] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.326980] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.328537] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.329200] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f] vm-test-run-nixbot-gitlab> gitlab # [ 0.329236] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.329317] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.330920] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.331555] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f] vm-test-run-nixbot-gitlab> gitlab # [ 0.331590] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.331672] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.331707] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.333420] pci 0000:00:08.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.334102] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.334183] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.335792] pci 0000:00:09.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.336434] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.336515] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.337939] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.338595] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.339409] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.340156] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] vm-test-run-nixbot-gitlab> gitlab # [ 0.340192] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.340273] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.341922] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.342549] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] vm-test-run-nixbot-gitlab> gitlab # [ 0.342584] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.342664] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.344604] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.344623] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.344633] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.344776] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.344908] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.345081] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.345228] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.345380] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.345548] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.345699] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.345853] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.346011] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.346192] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.346325] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.346379] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.346432] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.346485] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.346540] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.346595] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.346650] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.346704] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.346773] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.346831] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.346886] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.346953] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.347009] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.347063] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.347116] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.347179] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.347246] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.347303] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.347356] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.347410] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.347464] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.347530] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.347545] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.347554] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.351828] pci 0000:00:0a.0: enabling device (0000 -> 0002) vm-test-run-nixbot-gitlab> gitlab # [ 0.383229] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.387549] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.391997] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.398310] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.402656] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.406669] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.413012] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.417291] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) vm-test-run-nixbot-gitlab> gitlab # [ 0.420999] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) vm-test-run-nixbot-gitlab> gitlab # [ 0.427174] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.431927] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.443992] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.447697] msm_serial: driver initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.448092] SuperH (H)SCI(F) driver initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.448189] STM32 USART driver initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.494946] loop: module loaded vm-test-run-nixbot-gitlab> gitlab # [ 0.495646] virtio_blk virtio4: 4/0/0 default/read/poll queues vm-test-run-nixbot-gitlab> gitlab # [ 0.501187] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) vm-test-run-nixbot-gitlab> gitlab # [ 0.506003] virtio_blk virtio5: 4/0/0 default/read/poll queues vm-test-run-nixbot-gitlab> gitlab # [ 0.511404] virtio_blk virtio5: [vdb] 11828800 512-byte logical blocks (6.06 GB/5.64 GiB) vm-test-run-nixbot-gitlab> gitlab # [ 0.517120] megasas: 07.734.00.00-rc1 vm-test-run-nixbot-gitlab> gitlab # [ 0.519360] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.526138] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.526183] Intel/Sharp Extended Query Table at 0x0031 vm-test-run-nixbot-gitlab> gitlab # [ 0.531595] Using buffer write method vm-test-run-nixbot-gitlab> gitlab # [ 0.531745] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.536341] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.536381] Intel/Sharp Extended Query Table at 0x0031 vm-test-run-nixbot-gitlab> gitlab # [ 0.541357] Using buffer write method vm-test-run-nixbot-gitlab> gitlab # [ 0.541395] Concatenating MTD devices: vm-test-run-nixbot-gitlab> gitlab # [ 0.541402] (0): "0.flash" vm-test-run-nixbot-gitlab> gitlab # [ 0.541408] (1): "0.flash" vm-test-run-nixbot-gitlab> gitlab # [ 0.541414] into device "0.flash" vm-test-run-nixbot-gitlab> gitlab # [ 0.669397] Freeing initrd memory: 25372K vm-test-run-nixbot-gitlab> gitlab # [ 0.682688] tun: Universal TUN/TAP device driver, 1.6 vm-test-run-nixbot-gitlab> gitlab # [ 0.687122] thunder_xcv, ver 1.0 vm-test-run-nixbot-gitlab> gitlab # [ 0.687178] thunder_bgx, ver 1.0 vm-test-run-nixbot-gitlab> gitlab # [ 0.687208] nicpf, ver 1.0 vm-test-run-nixbot-gitlab> gitlab # [ 0.687795] e1000: Intel(R) PRO/1000 Network Driver vm-test-run-nixbot-gitlab> gitlab # [ 0.687801] e1000: Copyright (c) 1999-2006 Intel Corporation. vm-test-run-nixbot-gitlab> gitlab # [ 0.687831] e1000e: Intel(R) PRO/1000 Network Driver vm-test-run-nixbot-gitlab> gitlab # [ 0.687838] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. vm-test-run-nixbot-gitlab> gitlab # [ 0.687864] igb: Intel(R) Gigabit Ethernet Network Driver vm-test-run-nixbot-gitlab> gitlab # [ 0.687868] igb: Copyright (c) 2007-2014 Intel Corporation. vm-test-run-nixbot-gitlab> gitlab # [ 0.687894] igbvf: Intel(R) Gigabit Virtual Function Network Driver vm-test-run-nixbot-gitlab> gitlab # [ 0.687899] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. vm-test-run-nixbot-gitlab> gitlab # [ 0.688032] sky2: driver version 1.30 vm-test-run-nixbot-gitlab> gitlab # [ 0.689843] usbcore: registered new interface driver usb-storage vm-test-run-nixbot-gitlab> gitlab # [ 0.689925] usbcore: registered new interface driver usbserial_generic vm-test-run-nixbot-gitlab> gitlab # [ 0.689947] usbserial: USB Serial support registered for generic vm-test-run-nixbot-gitlab> gitlab # [ 0.690555] hv_vmbus: registering driver hyperv_keyboard vm-test-run-nixbot-gitlab> gitlab # [ 0.690883] ehci-pci 0000:00:0a.0: EHCI Host Controller vm-test-run-nixbot-gitlab> gitlab # [ 0.690963] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1 vm-test-run-nixbot-gitlab> gitlab # [ 0.691658] rtc-pl031 9010000.pl031: registered as rtc0 vm-test-run-nixbot-gitlab> gitlab # [ 0.691685] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000 vm-test-run-nixbot-gitlab> gitlab # [ 0.691694] rtc-pl031 9010000.pl031: setting system clock to 2026-07-25T11:25:40 UTC (1784978740) vm-test-run-nixbot-gitlab> gitlab # [ 0.692041] i2c_dev: i2c /dev entries driver vm-test-run-nixbot-gitlab> gitlab # [ 0.694828] sdhci: Secure Digital Host Controller Interface driver vm-test-run-nixbot-gitlab> gitlab # [ 0.694835] sdhci: Copyright(c) Pierre Ossman vm-test-run-nixbot-gitlab> gitlab # [ 0.695116] Synopsys Designware Multimedia Card Interface Driver vm-test-run-nixbot-gitlab> gitlab # [ 0.695486] sdhci-pltfm: SDHCI platform and OF driver helper vm-test-run-nixbot-gitlab> gitlab # [ 0.697257] hid: raw HID events driver (C) Jiri Kosina vm-test-run-nixbot-gitlab> gitlab # [ 0.697646] usbcore: registered new interface driver usbhid vm-test-run-nixbot-gitlab> gitlab # [ 0.697653] usbhid: USB HID core driver vm-test-run-nixbot-gitlab> gitlab # [ 0.717148] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00 vm-test-run-nixbot-gitlab> gitlab # [ 0.718411] hub 1-0:1.0: USB hub found vm-test-run-nixbot-gitlab> gitlab # [ 0.718413] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available vm-test-run-nixbot-gitlab> gitlab # [ 0.718546] hub 1-0:1.0: 6 ports detected vm-test-run-nixbot-gitlab> gitlab # [ 0.720507] drop_monitor: Initializing network drop monitor service vm-test-run-nixbot-gitlab> gitlab # [ 0.720717] NET: Registered PF_INET6 protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.722017] Segment Routing with IPv6 vm-test-run-nixbot-gitlab> gitlab # [ 0.722032] In-situ OAM (IOAM) with IPv6 vm-test-run-nixbot-gitlab> gitlab # [ 0.722056] NET: Registered PF_PACKET protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.722269] 9pnet: Installing 9P2000 support vm-test-run-nixbot-gitlab> gitlab # [ 0.724175] Key type dns_resolver registered vm-test-run-nixbot-gitlab> gitlab # [ 0.730433] registered taskstats version 1 vm-test-run-nixbot-gitlab> gitlab # [ 0.730668] Loading compiled-in X.509 certificates vm-test-run-nixbot-gitlab> gitlab # [ 0.739788] Demotion targets for Node 0: null vm-test-run-nixbot-gitlab> gitlab # [ 0.739947] Key type .fscrypt registered vm-test-run-nixbot-gitlab> gitlab # [ 0.739951] Key type fscrypt-provisioning registered vm-test-run-nixbot-gitlab> gitlab # [ 0.740071] ima: No TPM chip found, activating TPM-bypass! vm-test-run-nixbot-gitlab> gitlab # [ 0.740089] ima: Allocated hash algorithm: sha1 vm-test-run-nixbot-gitlab> gitlab # [ 0.740111] ima: No architecture policies found vm-test-run-nixbot-gitlab> gitlab # [ 0.741020] input: gpio-keys as /devices/platform/gpio-keys/input/input0 vm-test-run-nixbot-gitlab> gitlab # [ 0.758646] clk: Disabling unused clocks vm-test-run-nixbot-gitlab> gitlab # [ 0.758665] PM: genpd: Disabling unused power domains vm-test-run-nixbot-gitlab> gitlab # [ 0.762561] Freeing unused kernel memory: 4736K vm-test-run-nixbot-gitlab> gitlab # [ 0.762749] Run /init as init process vm-test-run-nixbot-gitlab> gitlab # [ 0.794342] systemd[1]: Successfully made /usr/ read-only. vm-test-run-nixbot-gitlab> gitlab # [ 0.965117] usb 1-1: new high-speed USB device number 2 using ehci-pci vm-test-run-nixbot-gitlab> gitlab # [ 1.120031] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 vm-test-run-nixbot-gitlab> gitlab # [ 1.128671] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) vm-test-run-nixbot-gitlab> gitlab # [ 1.128735] systemd[1]: Detected virtualization qemu. vm-test-run-nixbot-gitlab> gitlab # [ 1.128808] systemd[1]: Detected architecture arm64. vm-test-run-nixbot-gitlab> gitlab # [ 1.128824] systemd[1]: Running in initrd. vm-test-run-nixbot-gitlab> gitlab # [ 1.129925] systemd[1]: Initializing machine ID from random generator. vm-test-run-nixbot-gitlab> gitlab # [ 1.130198] systemd[1]: Hostname set to . vm-test-run-nixbot-gitlab> gitlab # [ 1.261810] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0 vm-test-run-nixbot-gitlab> gitlab # [ 1.280271] systemd[1]: Queued start job for default target Initrd Default Target. vm-test-run-nixbot-gitlab> gitlab # [ 1.389146] usb 1-2: new high-speed USB device number 3 using ehci-pci vm-test-run-nixbot-gitlab> gitlab # [ 1.527956] systemd[1]: Created slice Slice /system/modprobe. vm-test-run-nixbot-gitlab> gitlab # [ 1.528519] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 1.528591] systemd[1]: Expecting device /dev/disk/by-label/nix-store... vm-test-run-nixbot-gitlab> gitlab # [ 1.528617] systemd[1]: Expecting device /dev/disk/by-label/nixos... vm-test-run-nixbot-gitlab> gitlab # [ 1.528671] systemd[1]: Reached target Path Units. vm-test-run-nixbot-gitlab> gitlab # [ 1.528709] systemd[1]: Reached target Slice Units. vm-test-run-nixbot-gitlab> gitlab # [ 1.528751] systemd[1]: Reached target Swaps. vm-test-run-nixbot-gitlab> gitlab # [ 1.528785] systemd[1]: Reached target Timer Units. vm-test-run-nixbot-gitlab> gitlab # [ 1.529174] systemd[1]: Listening on D-Bus System Message Bus Socket. vm-test-run-nixbot-gitlab> gitlab # [ 1.529541] systemd[1]: Listening on Journal Socket (/dev/log). vm-test-run-nixbot-gitlab> gitlab # [ 1.529865] systemd[1]: Listening on Journal Sockets. vm-test-run-nixbot-gitlab> gitlab # [ 1.530166] systemd[1]: Listening on udev Control Socket. vm-test-run-nixbot-gitlab> gitlab # [ 1.530340] systemd[1]: Listening on udev Kernel Socket. vm-test-run-nixbot-gitlab> gitlab # [ 1.530379] systemd[1]: Reached target Socket Units. vm-test-run-nixbot-gitlab> gitlab # [ 1.538016] systemd[1]: Starting Create List of Static Device Nodes... vm-test-run-nixbot-gitlab> gitlab # [ 1.546251] systemd[1]: Starting Load Kernel Module 9pnet_virtio... vm-test-run-nixbot-gitlab> gitlab # [ 1.546617] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs vm-test-run-nixbot-gitlab> gitlab # [ 1.554649] systemd[1]: Mounting Kernel Configuration File System... vm-test-run-nixbot-gitlab> gitlab # [ 1.579512] systemd[1]: Starting Journal Service... vm-test-run-nixbot-gitlab> gitlab # [ 1.579991] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 vm-test-run-nixbot-gitlab> gitlab # [ 1.580455] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0 vm-test-run-nixbot-gitlab> gitlab # [ 1.587723] systemd[1]: Starting Load Kernel Modules... vm-test-run-nixbot-gitlab> gitlab # [ 1.587870] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 1.594496] systemd[1]: Starting Coldplug All udev Devices... vm-test-run-nixbot-gitlab> gitlab # [ 1.614085] systemd[1]: Finished Create List of Static Device Nodes. vm-test-run-nixbot-gitlab> gitlab # [ 1.616729] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 1.621866] systemd[1]: Finished Load Kernel Module 9pnet_virtio. vm-test-run-nixbot-gitlab> gitlab # [ 1.629487] systemd[1]: Mounted Kernel Configuration File System. vm-test-run-nixbot-gitlab> gitlab # [ 1.636298] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... vm-test-run-nixbot-gitlab> gitlab # [ 1.643134] systemd-journald[156]: Collecting audit messages is disabled. vm-test-run-nixbot-gitlab> gitlab # [ 1.646872] [drm] pci: virtio-gpu-pci detected at 0000:00:09.0 vm-test-run-nixbot-gitlab> gitlab # [ 1.647333] [drm] features: -virgl +edid -resource_blob -host_visible vm-test-run-nixbot-gitlab> gitlab # [ 1.647338] [drm] features: -context_init vm-test-run-nixbot-gitlab> gitlab # [ 1.654507] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. vm-test-run-nixbot-gitlab> gitlab # [ 1.655906] [drm] number of scanouts: 1 vm-test-run-nixbot-gitlab> gitlab # [ 1.655942] [drm] number of cap sets: 0 vm-test-run-nixbot-gitlab> gitlab # [ 1.661113] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev vm-test-run-nixbot-gitlab> gitlab # [ 1.664706] virtio-pci 0000:00:09.0: [drm] Registered 1 planes with drm panic vm-test-run-nixbot-gitlab> gitlab # [ 1.664736] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:09.0 on minor 0 vm-test-run-nixbot-gitlab> gitlab # [ 1.668327] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. vm-test-run-nixbot-gitlab> gitlab # [ 1.677679] Console: switching to colour frame buffer device 160x50 vm-test-run-nixbot-gitlab> gitlab # [ 1.684520] virtio-pci 0000:00:09.0: [drm] fb0: virtio_gpudrmfb frame buffer device vm-test-run-nixbot-gitlab> gitlab # [ 1.689894] systemd[1]: Starting Create Static Device Nodes in /dev... vm-test-run-nixbot-gitlab> gitlab # [ 1.709591] systemd[1]: Finished Load Kernel Modules. vm-test-run-nixbot-gitlab> gitlab # [ 1.719138] systemd[1]: Starting Apply Kernel Variables... vm-test-run-nixbot-gitlab> gitlab # [ 1.729471] systemd[1]: Finished Create Static Device Nodes in /dev. vm-test-run-nixbot-gitlab> gitlab # [ 1.730086] systemd[1]: Reached target Preparation for Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 1.730516] systemd[1]: Reached target Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 1.748550] systemd-modules-load[158]: Using 4 probe threads vm-test-run-nixbot-gitlab> gitlab # [ 1.750569] systemd-modules-load[158]: Module 'virtio_balloon' is built in vm-test-run-nixbot-gitlab> gitlab # [ 1.752499] systemd-modules-load[158]: Module 'virtio_console' is built in vm-test-run-nixbot-gitlab> gitlab # [ 1.754297] systemd-modules-load[158]: Module 'virtio_rng' is built in vm-test-run-nixbot-gitlab> gitlab # [ 1.755993] systemd-modules-load[158]: Inserted module 'dm_mod'[ 1.761304] systemd[1]: Starting Rule-based Manager for Device Events and Files... vm-test-run-nixbot-gitlab> gitlab # [ 1.761953] systemd[1]: Started Journal Service. vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # [ 1.766835] systemd-modules-load[158]: Inserted module 'virtio_gpu' vm-test-run-nixbot-gitlab> gitlab # [ 1.770016] systemd[1]: Starting Create System Files and Directories... vm-test-run-nixbot-gitlab> gitlab # [ 1.785313] systemd[1]: Finished Apply Kernel Variables. vm-test-run-nixbot-gitlab> gitlab # [ 1.813760] systemd-udevd[183]: Using default interface naming scheme 'v260'. vm-test-run-nixbot-gitlab> gitlab # [ 1.816503] systemd[1]: Finished Create System Files and Directories. vm-test-run-nixbot-gitlab> gitlab # [ 1.833438] systemd[1]: Started Rule-based Manager for Device Events and Files. vm-test-run-nixbot-gitlab> gitlab # [ 1.876889] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 1.927265] systemd-vconsole-setup[211]: Configuration of first virtual console was skipped, ignoring remaining ones. vm-test-run-nixbot-gitlab> gitlab # [ 1.932515] systemd[1]: Finished Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 2.328200] systemd[1]: Finished Coldplug All udev Devices. vm-test-run-nixbot-gitlab> gitlab # [ 2.329209] systemd[1]: Reached target System Initialization. vm-test-run-nixbot-gitlab> gitlab # [ 2.330026] systemd[1]: Reached target Basic System. vm-test-run-nixbot-gitlab> gitlab # [ 2.532318] (udev-worker)[199]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 2.546792] (udev-worker)[223]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. vm-test-run-nixbot-gitlab> gitlab # [ 2.552226] systemd[1]: Found device /dev/disk/by-label/nix-store. vm-test-run-nixbot-gitlab> gitlab # [ 2.557421] (udev-worker)[223]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 2.580923] systemd[1]: Found device /dev/disk/by-label/nixos. vm-test-run-nixbot-gitlab> gitlab # [ 2.584086] systemd[1]: Reached target Initrd Root Device. vm-test-run-nixbot-gitlab> gitlab # [ 2.588878] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... vm-test-run-nixbot-gitlab> gitlab # [ 2.625103] systemd-fsck[236]: nixos: clean, 12/65536 files, 13019/262144 blocks vm-test-run-nixbot-gitlab> gitlab # [ 2.632524] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. vm-test-run-nixbot-gitlab> gitlab # [ 2.639767] systemd[1]: Mounting /sysroot... vm-test-run-nixbot-gitlab> gitlab # [ 2.676194] EXT4-fs (vda): mounted filesystem 3a2117c2-08df-4e8c-b828-3fdf80aec035 r/w with ordered data mode. Quota mode: none. vm-test-run-nixbot-gitlab> gitlab # [ 2.677781] systemd[1]: Mounted /sysroot. vm-test-run-nixbot-gitlab> gitlab # [ 2.678701] systemd[1]: Reached target Initrd Root File System. vm-test-run-nixbot-gitlab> gitlab # [ 2.684927] systemd[1]: Mounting /sysroot/nix/.ro-store... vm-test-run-nixbot-gitlab> gitlab # [ 2.702879] systemd[1]: Mounting /sysroot/nix/.rw-store... vm-test-run-nixbot-gitlab> gitlab # [ 2.709804] systemd[1]: Mounting /sysroot/run... vm-test-run-nixbot-gitlab> gitlab # [ 2.724038] erofs (device vdb): mounted with root inode @ nid 128. vm-test-run-nixbot-gitlab> gitlab # [ 2.722204] systemd[1]: Mounting /sysroot/tmp/shared... vm-test-run-nixbot-gitlab> gitlab # [ 2.727036] systemd[1]: Mounting /sysroot/tmp/xchg... vm-test-run-nixbot-gitlab> gitlab # [ 2.742351] systemd[1]: Starting Mountpoints Configured in the Real Root... vm-test-run-nixbot-gitlab> gitlab # [ 2.747226] systemd[1]: Mounted /sysroot/nix/.ro-store. vm-test-run-nixbot-gitlab> gitlab # [ 2.756238] systemd[1]: Mounted /sysroot/nix/.rw-store. vm-test-run-nixbot-gitlab> gitlab # [ 2.758151] systemd[1]: Mounted /sysroot/run. vm-test-run-nixbot-gitlab> gitlab # [ 2.760501] systemd-sysroot-fstab-check[258]: /sysroot should be mounted in the initrd, will request daemon-reload. vm-test-run-nixbot-gitlab> gitlab # [ 2.762058] systemd[1]: Mounted /sysroot/tmp/shared. vm-test-run-nixbot-gitlab> gitlab # [ 2.762893] systemd[1]: Mounted /sysroot/tmp/xchg. vm-test-run-nixbot-gitlab> gitlab # [ 2.765946] systemd[1]: Starting rw-sysroot-nix-store.service... vm-test-run-nixbot-gitlab> gitlab # [ 2.767599] systemd[1]: Reload requested from client PID 258 ('systemd-sysroot') (unit initrd-parse-etc.service)... vm-test-run-nixbot-gitlab> gitlab # [ 2.769396] systemd[1]: Reloading... vm-test-run-nixbot-gitlab> gitlab # [ 3.091697] systemd[1]: Reloading finished in 324 ms. vm-test-run-nixbot-gitlab> gitlab # [ 3.141105] systemd-sysroot-fstab-check[258]: Requesting initrd-fs.target/start/replace... vm-test-run-nixbot-gitlab> gitlab # [ 3.234821] (udev-worker)[222]: mtd0ro: Failed to find and pin callout binary "/nix/store/3x4plcsrj2r4mmzis46wrmyka8g4jhi8-systemd-260.1/lib/udev/mtd_probe": No such file or directory vm-test-run-nixbot-gitlab> gitlab # [ 3.238473] (udev-worker)[222]: mtd0ro: /nix/store/3x4plcsrj2r4mmzis46wrmyka8g4jhi8-systemd-260.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory vm-test-run-nixbot-gitlab> gitlab # [ 3.255898] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.257779] systemd[1]: Finished rw-sysroot-nix-store.service. vm-test-run-nixbot-gitlab> gitlab # [ 3.265459] systemd-sysroot-fstab-check[258]: Requesting swap.target/start/replace... vm-test-run-nixbot-gitlab> gitlab # [ 3.270702] systemd[1]: initrd-parse-etc.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.273235] systemd[1]: Finished Mountpoints Configured in the Real Root. vm-test-run-nixbot-gitlab> gitlab # [ 3.274398] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. vm-test-run-nixbot-gitlab> gitlab # [ 3.285745] systemd[1]: Starting rw-sysroot-nix-store.service... vm-test-run-nixbot-gitlab> gitlab # [ 3.286833] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.288117] systemd[1]: Stopped Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 3.289838] systemd[1]: Stopping Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 3.307780] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 3.311198] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.314260] systemd[1]: Finished rw-sysroot-nix-store.service. vm-test-run-nixbot-gitlab> gitlab # [ 3.339741] systemd-vconsole-setup[487]: Configuration of first virtual console was skipped, ignoring remaining ones. vm-test-run-nixbot-gitlab> gitlab # [ 3.344511] systemd[1]: Finished Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 3.613022] systemd[1]: Mounting /sysroot/nix/store... vm-test-run-nixbot-gitlab> gitlab # [ 3.679310] systemd[1]: Mounted /sysroot/nix/store. vm-test-run-nixbot-gitlab> gitlab # [ 3.681906] systemd[1]: Reached target Initrd File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 3.689382] systemd[1]: Starting Find NixOS closure... vm-test-run-nixbot-gitlab> gitlab # [ 3.697399] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... vm-test-run-nixbot-gitlab> gitlab # [ 3.753371] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. vm-test-run-nixbot-gitlab> gitlab # [ 3.757524] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.781592] systemd[1]: Finished Find NixOS closure. vm-test-run-nixbot-gitlab> gitlab # [ 3.784118] systemd[1]: Reached target Initrd Default Target. vm-test-run-nixbot-gitlab> gitlab # [ 3.791989] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... vm-test-run-nixbot-gitlab> gitlab # [ 3.847312] systemd[1]: Stopped target Initrd Default Target. vm-test-run-nixbot-gitlab> gitlab # [ 3.850063] systemd[1]: Stopped target Basic System. vm-test-run-nixbot-gitlab> gitlab # [ 3.852387] systemd[1]: Stopped target Initrd Root Device. vm-test-run-nixbot-gitlab> gitlab # [ 3.854844] systemd[1]: Stopped target Path Units. vm-test-run-nixbot-gitlab> gitlab # [ 3.857105] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.860484] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 3.863883] systemd[1]: Stopped target Slice Units. vm-test-run-nixbot-gitlab> gitlab # [ 3.866263] systemd[1]: Stopped target Socket Units. vm-test-run-nixbot-gitlab> gitlab # [ 3.868547] systemd[1]: Stopped target System Initialization. vm-test-run-nixbot-gitlab> gitlab # [ 3.871034] systemd[1]: Stopped target Swaps. vm-test-run-nixbot-gitlab> gitlab # [ 3.873093] systemd[1]: Stopped target Timer Units. vm-test-run-nixbot-gitlab> gitlab # [ 3.875278] systemd[1]: dbus.socket: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.877930] systemd[1]: Closed D-Bus System Message Bus Socket. vm-test-run-nixbot-gitlab> gitlab # [ 3.880561] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.883800] systemd[1]: Stopped Find NixOS closure. vm-test-run-nixbot-gitlab> gitlab # [ 3.886156] systemd[1]: Starting Load Kernel Module 9pnet_virtio... vm-test-run-nixbot-gitlab> gitlab # [ 3.890002] systemd[1]: Starting rw-sysroot-nix-store.service... vm-test-run-nixbot-gitlab> gitlab # [ 3.892536] systemd[1]: systemd-sysctl.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.895213] systemd[1]: Stopped Apply Kernel Variables. vm-test-run-nixbot-gitlab> gitlab # [ 3.900886] systemd[1]: systemd-modules-load.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.903731] systemd[1]: Stopped Load Kernel Modules. vm-test-run-nixbot-gitlab> gitlab # [ 3.911740] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.915707] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. vm-test-run-nixbot-gitlab> gitlab # [ 3.921371] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.926016] systemd[1]: Stopped Create System Files and Directories. vm-test-run-nixbot-gitlab> gitlab # [ 3.931581] systemd[1]: Stopped target Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 3.934385] systemd[1]: Stopped target Preparation for Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 3.937585] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.940779] systemd[1]: Stopped Coldplug All udev Devices. vm-test-run-nixbot-gitlab> gitlab # [ 3.943253] systemd[1]: Stopping Rule-based Manager for Device Events and Files... vm-test-run-nixbot-gitlab> gitlab # [ 3.946572] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.949739] systemd[1]: Stopped Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 3.964538] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.969804] systemd[1]: Finished Load Kernel Module 9pnet_virtio. vm-test-run-nixbot-gitlab> gitlab # [ 3.980955] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.985984] systemd[1]: Finished rw-sysroot-nix-store.service. vm-test-run-nixbot-gitlab> gitlab # [ 3.997191] systemd[1]: systemd-udevd.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.002455] systemd[1]: Stopped Rule-based Manager for Device Events and Files. vm-test-run-nixbot-gitlab> gitlab # [ 4.005767] systemd[1]: systemd-udevd.service: Consumed 2.617s CPU time over 2.241s wall clock time, 33.1M memory peak. vm-test-run-nixbot-gitlab> gitlab # [ 4.022475] systemd[1]: initrd-cleanup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.027900] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. vm-test-run-nixbot-gitlab> gitlab # [ 4.043985] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.047236] systemd[1]: Closed udev Control Socket. vm-test-run-nixbot-gitlab> gitlab # [ 4.060594] systemd[1]: Starting Cleanup udev Database... vm-test-run-nixbot-gitlab> gitlab # [ 4.063066] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.066542] systemd[1]: Stopped Create Static Device Nodes in /dev. vm-test-run-nixbot-gitlab> gitlab # [ 4.070498] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.074143] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. vm-test-run-nixbot-gitlab> gitlab # [ 4.079736] systemd[1]: kmod-static-nodes.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.083220] systemd[1]: Stopped Create List of Static Device Nodes. vm-test-run-nixbot-gitlab> gitlab # [ 4.177494] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.180993] systemd[1]: Finished Cleanup udev Database. vm-test-run-nixbot-gitlab> gitlab # [ 4.191547] systemd[1]: Reached target Switch Root. vm-test-run-nixbot-gitlab> gitlab # [ 4.198986] systemd[1]: Starting NixOS Activation... vm-test-run-nixbot-gitlab> gitlab # [ 4.415535] initrd-nixos-activation-start[572]: booting system configuration /nix/store/rc187vq3l99p7w3vg2rwxlx21fkmlgvh-nixos-system-gitlab-test vm-test-run-nixbot-gitlab> gitlab # [ 4.479121] initrd-nixos-activation-start[572]: running activation script... vm-test-run-nixbot-gitlab> gitlab # [ 4.810114] initrd-nixos-activation-start[595]: setting up /etc... vm-test-run-nixbot-gitlab> gitlab # [ 4.991413] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.996088] systemd[1]: Finished NixOS Activation. vm-test-run-nixbot-gitlab> gitlab # [ 5.013694] systemd[1]: Starting Switch Root... vm-test-run-nixbot-gitlab> gitlab # [ 5.062242] systemd[1]: Switching root. vm-test-run-nixbot-gitlab> gitlab # [ 5.714050] systemd-journald[156]: Received SIGTERM from PID 1 (systemd). vm-test-run-nixbot-gitlab> gitlab # [ 6.448297] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) vm-test-run-nixbot-gitlab> gitlab # [ 6.448443] systemd[1]: Detected virtualization qemu. vm-test-run-nixbot-gitlab> gitlab # [ 6.448545] systemd[1]: Detected architecture arm64. vm-test-run-nixbot-gitlab> gitlab # [ 6.448730] systemd[1]: Detected first boot. vm-test-run-nixbot-gitlab> gitlab # [ 6.450449] systemd[1]: Initializing machine ID from random generator. vm-test-run-nixbot-gitlab> gitlab # [ 6.705489] systemd[1]: bpf-restrict-fs: LSM BPF program attached vm-test-run-nixbot-gitlab> gitlab # [ 6.815812] systemd[1]: Applying preset policy. vm-test-run-nixbot-gitlab> gitlab # [ 7.052073] systemd[1]: Populated /etc with preset unit settings. vm-test-run-nixbot-gitlab> gitlab # [ 7.244809] systemd[1]: initrd-switch-root.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.245995] systemd[1]: Stopped initrd-switch-root.service. vm-test-run-nixbot-gitlab> gitlab # [ 7.250145] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. vm-test-run-nixbot-gitlab> gitlab # [ 7.258412] systemd[1]: Created slice Slice /system/getty. vm-test-run-nixbot-gitlab> gitlab # [ 7.262619] systemd[1]: Created slice GitLab DevOps Platform Slice. vm-test-run-nixbot-gitlab> gitlab # [ 7.266713] systemd[1]: Created slice User and Session Slice. vm-test-run-nixbot-gitlab> gitlab # [ 7.268142] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 7.269812] systemd[1]: Started Forward Password Requests to Wall Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 7.270943] systemd[1]: Expecting device /dev/hvc0... vm-test-run-nixbot-gitlab> gitlab # [ 7.271995] systemd[1]: Expecting device /dev/ttyAMA0... vm-test-run-nixbot-gitlab> gitlab # [ 7.273131] systemd[1]: Reached target Local Encrypted Volumes. vm-test-run-nixbot-gitlab> gitlab # [ 7.274197] systemd[1]: Stopped target initrd-fs.target. vm-test-run-nixbot-gitlab> gitlab # [ 7.275165] systemd[1]: Stopped target initrd-root-fs.target. vm-test-run-nixbot-gitlab> gitlab # [ 7.276206] systemd[1]: Stopped target initrd-switch-root.target. vm-test-run-nixbot-gitlab> gitlab # [ 7.277240] systemd[1]: Reached target Virtual Machines and Containers. vm-test-run-nixbot-gitlab> gitlab # [ 7.278303] systemd[1]: Reached target Path Units. vm-test-run-nixbot-gitlab> gitlab # [ 7.279285] systemd[1]: Reached target Remote File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 7.280241] systemd[1]: Reached target Slice Units. vm-test-run-nixbot-gitlab> gitlab # [ 7.281369] systemd[1]: Reached target Swaps. vm-test-run-nixbot-gitlab> gitlab # [ 7.286171] systemd[1]: Listening on Process Core Dump Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.289531] systemd[1]: Listening on Credential Encryption/Decryption. vm-test-run-nixbot-gitlab> gitlab # [ 7.298665] systemd[1]: Starting Journal Log Access Socket... vm-test-run-nixbot-gitlab> gitlab # [ 7.301216] systemd[1]: Listening on Journal Audit Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.303144] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.304313] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 7.305845] systemd[1]: Listening on udev Control Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.317255] systemd[1]: Mounting Huge Pages File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.326407] systemd[1]: Mounting POSIX Message Queue File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.337219] systemd[1]: Mounting Kernel Debug File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.346968] systemd[1]: Mounting Kernel Trace File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.356039] systemd[1]: Starting Create List of Static Device Nodes... vm-test-run-nixbot-gitlab> gitlab # [ 7.373949] systemd[1]: Starting Load Kernel Module 9pnet_virtio... vm-test-run-nixbot-gitlab> gitlab # [ 7.375139] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs vm-test-run-nixbot-gitlab> gitlab # [ 7.383433] systemd[1]: Mounting Kernel Configuration File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.384498] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm vm-test-run-nixbot-gitlab> gitlab # [ 7.385535] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore vm-test-run-nixbot-gitlab> gitlab # [ 7.397343] systemd[1]: Starting Load Kernel Module fuse... vm-test-run-nixbot-gitlab> gitlab # [ 7.398369] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 vm-test-run-nixbot-gitlab> gitlab # [ 7.406999] systemd[1]: Starting Journal Service... vm-test-run-nixbot-gitlab> gitlab # [ 7.411464] systemd[1]: Starting Load Kernel Modules... vm-test-run-nixbot-gitlab> gitlab # [ 7.422326] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... vm-test-run-nixbot-gitlab> gitlab # [ 7.429516] systemd[1]: Starting Remount Root and Kernel File Systems... vm-test-run-nixbot-gitlab> gitlab # [ 7.430499] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 7.438660] fuse: init (API version 7.45) vm-test-run-nixbot-gitlab> gitlab # [ 7.445339] systemd[1]: Starting Coldplug All udev Devices... vm-test-run-nixbot-gitlab> gitlab # [ 7.456084] systemd-journald[804]: Collecting audit messages is enabled. vm-test-run-nixbot-gitlab> gitlab # [ 7.459138] systemd[1]: Listening on Journal Log Access Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.463145] systemd[1]: Mounted Huge Pages File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.464176] systemd[1]: Mounted POSIX Message Queue File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.465158] systemd[1]: Mounted Kernel Debug File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.466110] systemd[1]: Mounted Kernel Trace File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.467524] systemd[1]: Finished Create List of Static Device Nodes. vm-test-run-nixbot-gitlab> gitlab # [ 7.468757] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.469875] systemd[1]: Finished Load Kernel Module 9pnet_virtio. vm-test-run-nixbot-gitlab> gitlab # [ 7.470915] systemd[1]: Mounted Kernel Configuration File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.472040] systemd[1]: modprobe@fuse.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.473152] systemd[1]: Finished Load Kernel Module fuse. vm-test-run-nixbot-gitlab> gitlab # [ 7.479221] systemd[1]: Mounting FUSE Control File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.482083] systemd[1]: Queued start job for default target Multi-User System. vm-test-run-nixbot-gitlab> gitlab # [ 7.489074] systemd[1]: Unnecessary job was removed for /dev/vdb. vm-test-run-nixbot-gitlab> gitlab # [ 7.495404] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... vm-test-run-nixbot-gitlab> gitlab # [ 7.496464] systemd[1]: Started Journal Service. vm-test-run-nixbot-gitlab> gitlab # [ 7.495760] systemd[1]: systemd-journald.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.497391] systemd-modules-load[805]: Using 3 probe threads vm-test-run-nixbot-gitlab> gitlab # [ 7.498410] systemd-modules-load[805]: Module 'atkbd' is built in vm-test-run-nixbot-gitlab> gitlab # [ 7.499431] systemd-modules-load[805]: Module 'loop' is built in vm-test-run-nixbot-gitlab> gitlab # [ 7.500561] systemd-modules-load[805]: Inserted module 'tls' vm-test-run-nixbot-gitlab> gitlab # [ 7.501537] systemd[1]: Finished Load Kernel Modules. vm-test-run-nixbot-gitlab> gitlab # [ 7.513279] EXT4-fs (vda): re-mounted 3a2117c2-08df-4e8c-b828-3fdf80aec035. vm-test-run-nixbot-gitlab> gitlab # [ 7.516623] systemd-oomd[806]: No swap; memory pressure usage will be degraded vm-test-run-nixbot-gitlab> gitlab # [ 7.520874] systemd[1]: Starting Firewall... vm-test-run-nixbot-gitlab> gitlab # [ 7.525177] systemd[1]: Starting Apply Kernel Variables... vm-test-run-nixbot-gitlab> gitlab # [ 7.535799] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. vm-test-run-nixbot-gitlab> gitlab # [ 7.537611] systemd[1]: Finished Remount Root and Kernel File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 7.538992] systemd[1]: Mounted FUSE Control File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.556226] systemd[1]: Starting Flush Journal to Persistent Storage... vm-test-run-nixbot-gitlab> gitlab # [ 7.557596] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore vm-test-run-nixbot-gitlab> gitlab # [ 7.561054] systemd[1]: Starting Load/Save OS Random Seed... vm-test-run-nixbot-gitlab> gitlab # [ 7.564163] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 7.565512] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. vm-test-run-nixbot-gitlab> gitlab # [ 7.572256] systemd[1]: Starting Create Static Device Nodes in /dev... vm-test-run-nixbot-gitlab> gitlab # [ 7.576464] systemd[1]: Finished Apply Kernel Variables. vm-test-run-nixbot-gitlab> gitlab # [ 7.589865] systemd-journald[804]: Received client request to flush runtime journal. vm-test-run-nixbot-gitlab> gitlab # [ 7.621905] systemd[1]: Finished Load/Save OS Random Seed. vm-test-run-nixbot-gitlab> gitlab # [ 7.623497] systemd[1]: Finished Create Static Device Nodes in /dev. vm-test-run-nixbot-gitlab> gitlab # [ 7.626062] systemd[1]: Reached target First Boot Complete. vm-test-run-nixbot-gitlab> gitlab # [ 7.627245] systemd[1]: Reached target Preparation for Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 7.628671] systemd[1]: Starting Rule-based Manager for Device Events and Files... vm-test-run-nixbot-gitlab> gitlab # [ 7.630142] systemd[1]: Finished Flush Journal to Persistent Storage. vm-test-run-nixbot-gitlab> gitlab # [ 7.689922] systemd-udevd[850]: Using default interface naming scheme 'v260'. vm-test-run-nixbot-gitlab> gitlab # [ 7.733517] systemd[1]: Started Rule-based Manager for Device Events and Files. vm-test-run-nixbot-gitlab> gitlab # [ 8.036527] systemd[1]: Finished Firewall. vm-test-run-nixbot-gitlab> gitlab # [ 8.230602] systemd[1]: Finished Coldplug All udev Devices. vm-test-run-nixbot-gitlab> gitlab # [ 8.244877] systemd[1]: Mounting /run/wrappers... vm-test-run-nixbot-gitlab> gitlab # [ 8.274120] systemd[1]: Mounted /run/wrappers. vm-test-run-nixbot-gitlab> gitlab # [ 8.276424] systemd[1]: Reached target Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 8.278958] systemd[1]: Listening on Boot Loader Control Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 8.283785] systemd[1]: Starting register-nix-paths.service... vm-test-run-nixbot-gitlab> gitlab # [ 8.291376] systemd[1]: Starting Create SUID/SGID Wrappers... vm-test-run-nixbot-gitlab> gitlab # [ 8.294569] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. vm-test-run-nixbot-gitlab> gitlab # [ 8.311143] systemd[1]: Starting Save Transient machine-id to Disk... vm-test-run-nixbot-gitlab> gitlab # [ 8.317555] systemd[1]: Starting Create System Files and Directories... vm-test-run-nixbot-gitlab> gitlab # [ 8.347737] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs vm-test-run-nixbot-gitlab> gitlab # [ 8.380826] systemd[1]: Finished Create System Files and Directories. vm-test-run-nixbot-gitlab> gitlab # [ 8.422084] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 8.423446] systemd[1]: Finished Save Transient machine-id to Disk. vm-test-run-nixbot-gitlab> gitlab # [ 8.435009] systemd[1]: Starting Rebuild Journal Catalog... vm-test-run-nixbot-gitlab> gitlab # [ 8.438772] systemd[1]: Starting Record System Boot/Shutdown in UTMP... vm-test-run-nixbot-gitlab> gitlab # [ 8.455026] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. vm-test-run-nixbot-gitlab> gitlab # [ 8.470097] systemd[1]: Finished Record System Boot/Shutdown in UTMP. vm-test-run-nixbot-gitlab> gitlab # [ 8.487850] systemd[1]: Finished Rebuild Journal Catalog. vm-test-run-nixbot-gitlab> gitlab # [ 8.491906] systemd[1]: Starting Update is Completed... vm-test-run-nixbot-gitlab> gitlab # [ 8.506502] (udev-worker)[960]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 8.525531] systemd[1]: Finished Update is Completed. vm-test-run-nixbot-gitlab> gitlab # [ 8.548203] (udev-worker)[966]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. vm-test-run-nixbot-gitlab> gitlab # [ 8.551544] (udev-worker)[966]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 8.584555] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. vm-test-run-nixbot-gitlab> gitlab # [ 8.585974] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 8.587382] systemd[1]: Finished Create SUID/SGID Wrappers. vm-test-run-nixbot-gitlab> gitlab # [ 8.588394] systemd[1]: suid-sgid-wrappers.service: Consumed 191ms CPU time over 294ms wall clock time, 7.1M memory peak, 2.2M read from disk. vm-test-run-nixbot-gitlab> gitlab # [ 8.590218] systemd[1]: Condition check resulted in Virtio network device being skipped. vm-test-run-nixbot-gitlab> gitlab # [ 8.591413] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore vm-test-run-nixbot-gitlab> gitlab # [ 8.593824] systemd[1]: Starting Create SUID/SGID Wrappers... vm-test-run-nixbot-gitlab> gitlab # [ 8.596257] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. vm-test-run-nixbot-gitlab> gitlab # [ 8.599336] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 vm-test-run-nixbot-gitlab> gitlab # [ 8.602367] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore vm-test-run-nixbot-gitlab> gitlab # [ 8.604721] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 8.606915] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 8.667038] mousedev: PS/2 mouse device common for all mice vm-test-run-nixbot-gitlab> gitlab # [ 8.793972] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 8.795719] systemd[1]: Finished Create SUID/SGID Wrappers. vm-test-run-nixbot-gitlab> gitlab # [ 8.854337] systemd[1]: Finished register-nix-paths.service. vm-test-run-nixbot-gitlab> gitlab # [ 8.855458] systemd[1]: Reached target System Initialization. vm-test-run-nixbot-gitlab> gitlab # [ 8.856420] systemd[1]: Started Discard unused filesystem blocks once a week. vm-test-run-nixbot-gitlab> gitlab # [ 8.858573] systemd[1]: Started logrotate.timer. vm-test-run-nixbot-gitlab> gitlab # [ 8.859314] systemd[1]: Started Daily Cleanup of Temporary Directories. vm-test-run-nixbot-gitlab> gitlab # [ 8.861150] systemd[1]: Reached target Timer Units. vm-test-run-nixbot-gitlab> gitlab # [ 8.862080] systemd[1]: Listening on D-Bus System Message Bus Socket. vm-test-run-nixbot-gitlab> gitlab # [ 8.863842] systemd[1]: Listening on Nix Daemon Socket. vm-test-run-nixbot-gitlab> gitlab # [ 8.873889] systemd[1]: Starting nixbot CI service socket... vm-test-run-nixbot-gitlab> gitlab # [ 8.876187] systemd[1]: Listening on Hostname Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 8.877131] systemd[1]: Listening on nixbot CI service socket. vm-test-run-nixbot-gitlab> gitlab # [ 8.878414] systemd[1]: Reached target Socket Units. vm-test-run-nixbot-gitlab> gitlab # [ 8.881985] systemd[1]: Starting D-Bus System Message Bus... vm-test-run-nixbot-gitlab> gitlab # [ 8.925682] dbus-broker-launch[1123]: Looking up NSS user entry for 'systemd-timesync'... vm-test-run-nixbot-gitlab> gitlab # [ 8.928443] dbus-broker-launch[1123]: NSS returned no entry for 'systemd-timesync' vm-test-run-nixbot-gitlab> gitlab # [ 8.930138] dbus-broker-launch[1123]: Invalid user-name in /nix/store/ifm9k575wb3j66gvjx1xkdmmknzqr8v8-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" vm-test-run-nixbot-gitlab> gitlab # [ 8.935669] systemd[1]: Started D-Bus System Message Bus. vm-test-run-nixbot-gitlab> gitlab # [ 8.949105] dbus-broker-launch[1123]: Ready vm-test-run-nixbot-gitlab> gitlab # [ 8.949945] systemd[1]: Reached target Basic System. vm-test-run-nixbot-gitlab> gitlab # [ 8.954578] systemd[1]: Started backdoor.service. vm-test-run-nixbot-gitlab> gitlab # [ 8.956128] systemd[1]: Starting gitlab-config.service... vm-test-run-nixbot-gitlab> gitlab # [ 8.960992] systemd[1]: Starting Import lastlog data into lastlog2 database... vm-test-run-nixbot-gitlab> gitlab # [ 8.982431] systemd[1]: Starting Logrotate configuration check... vm-test-run-nixbot-gitlab> gitlab # [ 8.993255] systemd[1]: Starting Name Service Cache Daemon (nsncd)... vm-test-run-nixbot-gitlab> gitlab # [ 8.999406] logrotate[1128]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. vm-test-run-nixbot-gitlab> gitlab # [ 9.002315] logrotate[1128]: reading config file /nix/store/hk6ay5j24mh4h8mpr5rxlxxxp8v61fnj-logrotate.conf vm-test-run-nixbot-gitlab> gitlab # [ 9.003827] logrotate[1128]: note: 'monthly' overrides previously specified 'weekly' vm-test-run-nixbot-gitlab> gitlab # [ 9.005672] logrotate[1128]: Reading state from file: /var/lib/logrotate.status vm-test-run-nixbot-gitlab> gitlab # connecting to host... vm-test-run-nixbot-gitlab> gitlab # [ 9.008493] logrotate[1128]: state file /var/lib/logrotate.status does not exist vm-test-run-nixbot-gitlab> gitlab # [ 9.009573] logrotate[1128]: Allocating hash table for state file, size 64 entries vm-test-run-nixbot-gitlab> gitlab # [ 9.010714] logrotate[1128]: Handling 4 logs vm-test-run-nixbot-gitlab> gitlab # [ 9.011331] logrotate[1128]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 9.013885] logrotate[1128]: considering log /var/log/btmp vm-test-run-nixbot-gitlab> gitlab # [ 9.015145] logrotate[1128]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 9.015837] logrotate[1128]: Now: 2026-07-25 11:25 vm-test-run-nixbot-gitlab> gitlab # [ 9.016628] logrotate[1128]: Last rotated at 2026-07-25 11:00 vm-test-run-nixbot-gitlab> gitlab # [ 9.017443] logrotate[1128]: log does not need rotating (log has already been rotated) vm-test-run-nixbot-gitlab> gitlab # [ 9.018635] logrotate[1128]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 9.022493] logrotate[1128]: considering log /var/log/wtmp vm-test-run-nixbot-gitlab> gitlab # [ 9.023307] logrotate[1128]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 9.023944] logrotate[1128]: Now: 2026-07-25 11:25 vm-test-run-nixbot-gitlab> gitlab # [ 9.024853] logrotate[1128]: Last rotated at 2026-07-25 11:00 vm-test-run-nixbot-gitlab> gitlab # [ 9.025668] logrotate[1128]: log does not need rotating (log has already been rotated) vm-test-run-nixbot-gitlab> gitlab # [ 9.029859] logrotate[1128]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 9.032653] logrotate[1128]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1128) vm-test-run-nixbot-gitlab> gitlab # [ 9.033794] logrotate[1128]: considering log /var/gitlab/state/log/*.log vm-test-run-nixbot-gitlab> gitlab: Guest shell says: b'Spawning backdoor root shell...\n' vm-test-run-nixbot-gitlab> gitlab: connected to guest root shell vm-test-run-nixbot-gitlab> gitlab # [ 9.035141] logrotate[1128]: log /var/gitlab/state/log/*.log does not exist -- skipping vm-test-run-nixbot-gitlab> gitlab: (connecting took 9.52 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 9.036227] logrotate[1128]: Creating new state vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for the VM to finish booting, in 9.52 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 9.040244] logrotate[1128]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1128) vm-test-run-nixbot-gitlab> gitlab # [ 9.041389] logrotate[1128]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 9.042978] logrotate[1128]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1128) vm-test-run-nixbot-gitlab> gitlab # [ 9.043988] logrotate[1128]: considering log /var/log/nginx/*.log vm-test-run-nixbot-gitlab> gitlab # [ 9.046574] logrotate[1128]: log /var/log/nginx/*.log does not exist -- skipping vm-test-run-nixbot-gitlab> gitlab # [ 9.047587] logrotate[1128]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 9.048410] logrotate[1128]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1128) vm-test-run-nixbot-gitlab> gitlab # [ 9.052230] systemd[1]: Starting Post-Boot Actions... vm-test-run-nixbot-gitlab> gitlab # [ 9.053377] systemd[1]: Started Reset console on configuration changes. vm-test-run-nixbot-gitlab> gitlab # [ 9.056629] systemd[1]: Starting resolvconf update... vm-test-run-nixbot-gitlab> gitlab # [ 9.058813] nsncd[1129]: Jul 25 11:25:48.868 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" vm-test-run-nixbot-gitlab> gitlab # [ 9.064510] systemd[1]: Finished Logrotate configuration check. vm-test-run-nixbot-gitlab> gitlab # [ 9.065538] systemd[1]: Finished Post-Boot Actions. vm-test-run-nixbot-gitlab> gitlab # [ 9.068267] systemd[1]: Finished Import lastlog data into lastlog2 database. vm-test-run-nixbot-gitlab> gitlab # [ 9.069472] systemd[1]: Started Name Service Cache Daemon (nsncd). vm-test-run-nixbot-gitlab> gitlab # [ 9.070339] systemd[1]: Reached target Host and Network Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 9.071309] systemd[1]: Reached target User and Group Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 9.072991] systemd[1]: Starting User Login Management... vm-test-run-nixbot-gitlab> gitlab # [ 9.139315] systemd-logind[1165]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) vm-test-run-nixbot-gitlab> gitlab # [ 9.145639] systemd-logind[1165]: Watching system buttons on /dev/input/event0 (gpio-keys) vm-test-run-nixbot-gitlab> gitlab # [ 9.153491] systemd[1]: Stopped target Host and Network Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 9.154507] systemd[1]: Stopping Host and Network Name Lookups... vm-test-run-nixbot-gitlab> gitlab # [ 9.155393] systemd[1]: Stopped target User and Group Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 9.158263] systemd[1]: Stopping User and Group Name Lookups... vm-test-run-nixbot-gitlab> gitlab # [ 9.159218] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... vm-test-run-nixbot-gitlab> gitlab # [ 9.160218] systemd[1]: nscd.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 9.161072] systemd-logind[1165]: New seat seat0. vm-test-run-nixbot-gitlab> gitlab # [ 9.161785] systemd[1]: Stopped Name Service Cache Daemon (nsncd). vm-test-run-nixbot-gitlab> gitlab # [ 9.162708] systemd[1]: nscd.service: Consumed 25ms CPU time over 149ms wall clock time, 8.1M memory peak, 5.9M read from disk. vm-test-run-nixbot-gitlab> gitlab # [ 9.165664] systemd[1]: Started User Login Management. vm-test-run-nixbot-gitlab> gitlab # [ 9.166524] systemd[1]: Starting linger-users.service... vm-test-run-nixbot-gitlab> gitlab # [ 9.167358] systemd[1]: Starting Name Service Cache Daemon (nsncd)... vm-test-run-nixbot-gitlab> gitlab # [ 9.194523] systemd[1]: Finished resolvconf update. vm-test-run-nixbot-gitlab> gitlab # [ 9.195880] nsncd[1219]: Jul 25 11:25:49.007 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" vm-test-run-nixbot-gitlab> gitlab # [ 9.198257] systemd[1]: Started Name Service Cache Daemon (nsncd). vm-test-run-nixbot-gitlab> gitlab # [ 9.199120] systemd[1]: Reached target Preparation for Network. vm-test-run-nixbot-gitlab> gitlab # [ 9.200184] systemd[1]: Reached target Host and Network Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 9.201094] systemd[1]: Reached target User and Group Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 9.204071] systemd[1]: Starting DHCP Client... vm-test-run-nixbot-gitlab> gitlab # [ 9.206617] systemd[1]: Starting Address configuration of eth1... vm-test-run-nixbot-gitlab> gitlab # [ 9.216548] systemd[1]: Starting Extra networking commands.... vm-test-run-nixbot-gitlab> gitlab # [ 9.217512] systemd[1]: linger-users.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 9.219768] systemd[1]: Finished linger-users.service. vm-test-run-nixbot-gitlab> gitlab # [ 9.265766] network-addresses-eth1-start[1241]: adding address 192.168.1.1/24... done vm-test-run-nixbot-gitlab> gitlab # [ 9.272328] network-addresses-eth1-start[1241]: adding address 2001:db8:1::1/64... done vm-test-run-nixbot-gitlab> gitlab # [ 9.280128] systemd[1]: Finished Address configuration of eth1. vm-test-run-nixbot-gitlab> gitlab # [ 9.324087] systemd[1]: Finished Extra networking commands.. vm-test-run-nixbot-gitlab> gitlab # [ 9.326416] systemd[1]: Reached target Network. vm-test-run-nixbot-gitlab> gitlab # [ 9.332082] systemd[1]: Starting gitlab-workhorse.service... vm-test-run-nixbot-gitlab> gitlab # [ 9.343080] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:08.0/virtio7/input/input3 vm-test-run-nixbot-gitlab> gitlab # [ 9.344167] systemd[1]: Starting Nginx Web Server... vm-test-run-nixbot-gitlab> gitlab # [ 9.349066] dhcpcd[1252]: dhcpcd-10.3.2 starting vm-test-run-nixbot-gitlab> gitlab # [ 9.357613] dhcpcd[1302]: dev: loaded udev vm-test-run-nixbot-gitlab> gitlab # [ 9.360576] systemd[1]: Starting PostgreSQL Server... vm-test-run-nixbot-gitlab> gitlab # [ 9.378709] systemd[1]: Starting Redis Server - redis-gitlab... vm-test-run-nixbot-gitlab> gitlab # [ 9.385504] 8021q: 802.1Q VLAN Support v1.8 vm-test-run-nixbot-gitlab> gitlab # [ 9.385941] 8021q: adding VLAN 0 to HW filter on device eth1 vm-test-run-nixbot-gitlab> gitlab # [ 9.381990] systemd[1]: Starting Permit User Sessions... vm-test-run-nixbot-gitlab> gitlab # [ 9.422460] systemd[1]: Finished Permit User Sessions. vm-test-run-nixbot-gitlab> gitlab # [ 9.439050] systemd[1]: Started Getty on tty1. vm-test-run-nixbot-gitlab> gitlab # [ 9.440620] systemd[1]: Reached target Login Prompts. vm-test-run-nixbot-gitlab> gitlab # [ 9.482747] cfg80211: Loading compiled-in X.509 certificates for regulatory database vm-test-run-nixbot-gitlab> gitlab # [ 9.497510] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' vm-test-run-nixbot-gitlab> gitlab # [ 9.498115] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' vm-test-run-nixbot-gitlab> gitlab # [ 9.500184] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 vm-test-run-nixbot-gitlab> gitlab # [ 9.500562] cfg80211: failed to load regulatory.db vm-test-run-nixbot-gitlab> gitlab # [ 9.555995] 8021q: adding VLAN 0 to HW filter on device eth0 vm-test-run-nixbot-gitlab> gitlab # [ 9.552047] dhcpcd[1302]: eth0: waiting for carrier vm-test-run-nixbot-gitlab> gitlab # [ 9.556221] dhcpcd[1302]: eth0: carrier acquired vm-test-run-nixbot-gitlab> gitlab # [ 9.561544] dhcpcd[1302]: DUID 00:01:00:01:31:f7:59:bd:52:54:00:12:34:56 vm-test-run-nixbot-gitlab> gitlab # [ 9.564292] dhcpcd[1302]: eth0: IAID 00:12:34:56 vm-test-run-nixbot-gitlab> gitlab # [ 9.565355] dhcpcd[1302]: eth0: adding address fe80::5054:ff:fe12:3456 vm-test-run-nixbot-gitlab> gitlab # [ 9.613468] redis-server[1324]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. vm-test-run-nixbot-gitlab> gitlab # [ 9.620106] redis-server[1324]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo vm-test-run-nixbot-gitlab> gitlab # [ 9.621858] nginx-pre-start[1330]: nginx: the configuration file /nix/store/2526shpjv0bzachwf4qi968gxiiyv4kq-nginx.conf syntax is ok vm-test-run-nixbot-gitlab> gitlab # [ 9.623410] nginx-pre-start[1330]: nginx: configuration file /nix/store/2526shpjv0bzachwf4qi968gxiiyv4kq-nginx.conf test is successful vm-test-run-nixbot-gitlab> gitlab # [ 9.626938] redis-server[1324]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1324, just started vm-test-run-nixbot-gitlab> gitlab # [ 9.630421] systemd[1]: Started Nginx Web Server. vm-test-run-nixbot-gitlab> gitlab # [ 9.631402] redis-server[1324]: Configuration loaded vm-test-run-nixbot-gitlab> gitlab # [ 9.632340] systemd[1]: Started Redis Server - redis-gitlab. vm-test-run-nixbot-gitlab> gitlab # [ 9.633276] redis[1324]: monotonic clock: ARM CNTVCT @ 1000 ticks/us vm-test-run-nixbot-gitlab> gitlab # [ 9.634292] redis[1324]: Running mode=standalone, port=0. vm-test-run-nixbot-gitlab> gitlab # [ 9.635193] redis[1324]: Server initialized vm-test-run-nixbot-gitlab> gitlab # [ 9.635995] redis[1324]: Ready to accept connections unix vm-test-run-nixbot-gitlab> gitlab # [ 9.636941] redis[1324]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. vm-test-run-nixbot-gitlab> gitlab # [ 9.644732] postgresql-pre-start[1335]: The files belonging to this database system will be owned by user "postgres". vm-test-run-nixbot-gitlab> gitlab # [ 9.646772] postgresql-pre-start[1335]: This user must also own the server process. vm-test-run-nixbot-gitlab> gitlab # [ 9.653276] postgresql-pre-start[1335]: The database cluster will be initialized with locale "en_US.UTF-8". vm-test-run-nixbot-gitlab> gitlab # [ 9.654826] postgresql-pre-start[1335]: The default database encoding has accordingly been set to "UTF8". vm-test-run-nixbot-gitlab> gitlab # [ 9.657986] postgresql-pre-start[1335]: The default text search configuration will be set to "english". vm-test-run-nixbot-gitlab> gitlab # [ 9.659473] postgresql-pre-start[1335]: Data page checksums are disabled. vm-test-run-nixbot-gitlab> gitlab # [ 9.663313] postgresql-pre-start[1335]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok vm-test-run-nixbot-gitlab> gitlab # [ 9.665826] postgresql-pre-start[1335]: creating subdirectories ... ok vm-test-run-nixbot-gitlab> gitlab # [ 9.667221] postgresql-pre-start[1335]: selecting dynamic shared memory implementation ... posix vm-test-run-nixbot-gitlab> gitlab # [ 9.711744] postgresql-pre-start[1335]: selecting default "max_connections" ... 100 vm-test-run-nixbot-gitlab> gitlab # [ 9.731429] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 9.757493] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 9.758872] systemd[1]: Stopped Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 9.761715] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 9.783997] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. vm-test-run-nixbot-gitlab> gitlab # [ 9.791738] postgresql-pre-start[1335]: selecting default "shared_buffers" ... 128MB vm-test-run-nixbot-gitlab> gitlab # [ 9.827336] systemd-logind[1165]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) vm-test-run-nixbot-gitlab> gitlab # [ 9.862922] systemd-vconsole-setup[1364]: Configuration of first virtual console was skipped, ignoring remaining ones. vm-test-run-nixbot-gitlab> gitlab # [ 9.866252] systemd[1]: Finished Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 10.045517] systemd[1]: Started gitlab-workhorse.service. vm-test-run-nixbot-gitlab> gitlab # [ 10.251081] postgresql-pre-start[1335]: selecting default time zone ... UTC vm-test-run-nixbot-gitlab> gitlab # [ 10.253642] postgresql-pre-start[1335]: creating configuration files ... ok vm-test-run-nixbot-gitlab> gitlab # [ 10.454171] gitlab-workhorse[1369]: time="2026-07-25T11:25:50Z" level=info msg=Starting build_time=19700101.000000 version=18.11.4 vm-test-run-nixbot-gitlab> gitlab # [ 10.459586] gitlab-workhorse[1369]: time="2026-07-25T11:25:50Z" level=info msg="Using redis/go-redis" vm-test-run-nixbot-gitlab> gitlab # [ 10.467902] gitlab-workhorse[1369]: time="2026-07-25T11:25:50Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix vm-test-run-nixbot-gitlab> gitlab # [ 10.514519] postgresql-pre-start[1335]: running bootstrap script ... ok vm-test-run-nixbot-gitlab> gitlab # [ 10.854265] dhcpcd[1302]: eth0: soliciting a DHCP lease vm-test-run-nixbot-gitlab> gitlab # [ 10.869229] dhcpcd[1302]: eth0: offered 10.0.2.15 from 10.0.2.2 vm-test-run-nixbot-gitlab> gitlab # [ 10.892744] dhcpcd[1302]: eth0: probing address 10.0.2.15/24 vm-test-run-nixbot-gitlab> gitlab # [ 11.169867] postgresql-pre-start[1335]: performing post-bootstrap initialization ... ok vm-test-run-nixbot-gitlab> gitlab # [ 11.468325] postgresql-pre-start[1335]: syncing data to disk ... ok vm-test-run-nixbot-gitlab> gitlab # [ 11.471355] postgresql-pre-start[1335]: initdb: warning: enabling "trust" authentication for local connections vm-test-run-nixbot-gitlab> gitlab # [ 11.475844] postgresql-pre-start[1335]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. vm-test-run-nixbot-gitlab> gitlab # [ 11.483073] postgresql-pre-start[1335]: Success. You can now start the database server using: vm-test-run-nixbot-gitlab> gitlab # [ 11.489903] postgresql-pre-start[1335]: pg_ctl -D /var/lib/postgresql/17 -l logfile start vm-test-run-nixbot-gitlab> gitlab # [ 11.649137] postgres[1403]: [1403] LOG: starting PostgreSQL 17.10 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit vm-test-run-nixbot-gitlab> gitlab # [ 11.654924] postgres[1403]: [1403] LOG: listening on IPv6 address "::1", port 5432 vm-test-run-nixbot-gitlab> gitlab # [ 11.658460] postgres[1403]: [1403] LOG: listening on IPv4 address "127.0.0.1", port 5432 vm-test-run-nixbot-gitlab> gitlab # [ 11.662336] postgres[1403]: [1403] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" vm-test-run-nixbot-gitlab> gitlab # [ 11.671216] postgres[1410]: [1410] LOG: database system was shut down at 2026-07-25 11:25:50 GMT vm-test-run-nixbot-gitlab> gitlab # [ 11.675559] dhcpcd[1302]: eth0: soliciting an IPv6 router vm-test-run-nixbot-gitlab> gitlab # [ 11.678699] dhcpcd[1302]: eth0: Router Advertisement from fe80::2 vm-test-run-nixbot-gitlab> gitlab # [ 11.681696] dhcpcd[1302]: eth0: adding address fec0::5054:ff:fe12:3456/64 vm-test-run-nixbot-gitlab> gitlab # [ 11.684741] dhcpcd[1302]: eth0: adding route to fec0::/64 vm-test-run-nixbot-gitlab> gitlab # [ 11.687130] dhcpcd[1302]: eth0: adding default route via fe80::2 vm-test-run-nixbot-gitlab> gitlab # [ 11.694804] postgres[1403]: [1403] LOG: database system is ready to accept connections vm-test-run-nixbot-gitlab> gitlab # [ 11.705948] systemd[1]: Started PostgreSQL Server. vm-test-run-nixbot-gitlab> gitlab # [ 11.712148] systemd[1]: Starting PostgreSQL Setup Scripts... vm-test-run-nixbot-gitlab> gitlab # [ 11.987504] c0sj6d459nhf3wg9pm6ziy71l4581qq8-gitlab-config[1420]: mkdir -p /var/gitlab/state/home/.ssh: OK vm-test-run-nixbot-gitlab> gitlab # [ 11.992007] c0sj6d459nhf3wg9pm6ziy71l4581qq8-gitlab-config[1420]: chmod 700 /var/gitlab/state/home/.ssh: OK vm-test-run-nixbot-gitlab> gitlab # [ 12.032163] postgresql-setup-start[1425]: CREATE DATABASE vm-test-run-nixbot-gitlab> gitlab # [ 12.083697] postgresql-setup-start[1440]: CREATE ROLE vm-test-run-nixbot-gitlab> gitlab # [ 12.096306] systemd[1]: Finished gitlab-config.service. vm-test-run-nixbot-gitlab> gitlab # [ 12.101978] systemd[1]: Started gitaly.service. vm-test-run-nixbot-gitlab> gitlab # [ 12.108756] postgresql-setup-start[1444]: ALTER DATABASE vm-test-run-nixbot-gitlab> gitlab # [ 12.171473] postgresql-setup-start[1452]: CREATE ROLE vm-test-run-nixbot-gitlab> gitlab # [ 12.177003] systemd[1]: Finished PostgreSQL Setup Scripts. vm-test-run-nixbot-gitlab> gitlab # [ 12.179449] systemd[1]: Reached target PostgreSQL. vm-test-run-nixbot-gitlab> gitlab # [ 12.184294] systemd[1]: Starting gitlab-db-config.service... vm-test-run-nixbot-gitlab> gitlab # [ 12.190591] systemd[1]: Starting gitlab-postgresql.service... vm-test-run-nixbot-gitlab> gitlab # [ 12.437013] gitlab-postgresql-start[1462]: CREATE DATABASE vm-test-run-nixbot-gitlab> gitlab # [ 12.520418] gitlab-postgresql-start[1481]: CREATE EXTENSION vm-test-run-nixbot-gitlab> gitlab # [ 12.620990] gitlab-postgresql-start[1483]: CREATE EXTENSION vm-test-run-nixbot-gitlab> gitlab # [ 12.626771] systemd[1]: Finished gitlab-postgresql.service. vm-test-run-nixbot-gitlab> gitlab # [ 12.999493] gitaly[1447]: time="2026-07-25T11:25:52.811Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/krygc3bmd623znp2pddh3fv21q5f3nz2-gitaly.toml' and fix the errors that are printed." pid=1447 vm-test-run-nixbot-gitlab> gitlab # [ 13.016331] gitaly[1447]: time="2026-07-25T11:25:52.812Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=1447 vm-test-run-nixbot-gitlab> gitlab # [ 13.018429] gitaly[1447]: time="2026-07-25T11:25:52.812Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=1447 vm-test-run-nixbot-gitlab> gitlab # [ 13.021410] gitaly[1447]: time="2026-07-25T11:25:52.812Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=1447 vm-test-run-nixbot-gitlab> gitlab # [ 13.023907] gitaly[1447]: time="2026-07-25T11:25:52.812Z" level=info msg="Starting Gitaly" pid=1447 version=18.11.4 vm-test-run-nixbot-gitlab> gitlab # [ 13.275374] gitaly[1447]: time="2026-07-25T11:25:53.088Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=274 pid=1447 vm-test-run-nixbot-gitlab> gitlab # [ 13.279968] gitaly[1447]: time="2026-07-25T11:25:53.088Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=1447 vm-test-run-nixbot-gitlab> gitlab # [ 13.283819] gitaly[1447]: time="2026-07-25T11:25:53.089Z" level=info msg="finished initializing command factory" duration_ms=0 pid=1447 vm-test-run-nixbot-gitlab> gitlab # [ 13.288623] gitaly[1447]: time="2026-07-25T11:25:53.089Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1447/git-exec-1217201969.d/git pid=1447 vm-test-run-nixbot-gitlab> gitlab # [ 13.294389] gitaly[1447]: time="2026-07-25T11:25:53.094Z" level=info msg="finished detecting git version" duration_ms=5 pid=1447 vm-test-run-nixbot-gitlab> gitlab # [ 13.298742] gitaly[1447]: time="2026-07-25T11:25:53.094Z" level=info msg="using Git version" pid=1447 version=2.53.g vm-test-run-nixbot-gitlab> gitlab # [ 13.302759] gitaly[1447]: time="2026-07-25T11:25:53.094Z" level=info msg="clearing disk cache object folder" pid=1447 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 13.307550] gitaly[1447]: time="2026-07-25T11:25:53.094Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache4004159446 pid=1447 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 13.314580] gitaly[1447]: time="2026-07-25T11:25:53.094Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache4004159446 pid=1447 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 13.322706] gitaly[1447]: time="2026-07-25T11:25:53.094Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=1447 vm-test-run-nixbot-gitlab> gitlab # [ 13.329964] gitaly[1447]: time="2026-07-25T11:25:53.094Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=1447 vm-test-run-nixbot-gitlab> gitlab # [ 13.337179] gitaly[1447]: time="2026-07-25T11:25:53.095Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache4004159446 clear_duration_ms=0 pid=1447 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 13.344783] gitaly[1447]: time="2026-07-25T11:25:53.110Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=1447 vm-test-run-nixbot-gitlab> gitlab # [ 13.350634] gitaly[1447]: time="2026-07-25T11:25:53.110Z" level=info msg="counting repositories" pid=1447 vm-test-run-nixbot-gitlab> gitlab # [ 13.354412] gitaly[1447]: time="2026-07-25T11:25:53.110Z" level=info msg="starting to count repositories" pid=1447 storage_path=/var/gitlab/state/repositories vm-test-run-nixbot-gitlab> gitlab # [ 13.359804] gitaly[1447]: time="2026-07-25T11:25:53.110Z" level=info msg="finished tempdir cleaner walk" pid=1447 storage=default time_ms=0 vm-test-run-nixbot-gitlab> gitlab # [ 13.366081] gitaly[1447]: time="2026-07-25T11:25:53.110Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1447 schema=unix vm-test-run-nixbot-gitlab> gitlab # [ 13.370900] gitaly[1447]: time="2026-07-25T11:25:53.110Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1447 storage_path=/var/gitlab/state/repositories vm-test-run-nixbot-gitlab> gitlab # [ 13.376297] gitaly[1447]: time="2026-07-25T11:25:53.110Z" level=info msg="completed counting all repositories" counting_duration=0 pid=1447 vm-test-run-nixbot-gitlab> gitlab # [ 13.380396] gitaly[1447]: time="2026-07-25T11:25:53.110Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1447/sock.d/intern pid=1447 schema=unix vm-test-run-nixbot-gitlab> gitlab # [ 13.385221] gitaly[1447]: time="2026-07-25T11:25:53.112Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=1447 vm-test-run-nixbot-gitlab> gitlab # [ 13.389507] gitaly[1447]: time="2026-07-25T11:25:53.112Z" level=info msg="Started Gitaly" duration_ms=299 pid=1447 vm-test-run-nixbot-gitlab> gitlab # [ 13.393346] gitaly[1447]: time="2026-07-25T11:25:53.112Z" level=info msg="maintenance: daily scheduled" pid=1447 scheduled="2026-07-25 12:00:00 +0000 UTC" vm-test-run-nixbot-gitlab> gitlab # [ 13.592690] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1457]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 13.603055] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1457]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 13.612129] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1457]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 15.793681] dhcpcd[1302]: eth0: leased 10.0.2.15 for 86400 seconds vm-test-run-nixbot-gitlab> gitlab # [ 15.794031] dhcpcd[1302]: eth0: adding route to 10.0.2.0/24 vm-test-run-nixbot-gitlab> gitlab # [ 15.794171] dhcpcd[1302]: eth0: adding default route via 10.0.2.2 vm-test-run-nixbot-gitlab> gitlab # [ 15.972824] systemd[1]: Started DHCP Client. vm-test-run-nixbot-gitlab> gitlab # [ 15.975610] systemd[1]: Reached target Network is Online. vm-test-run-nixbot-gitlab> gitlab # [ 16.937139] gitaly[1447]: time="2026-07-25T11:25:56.749Z" level=info msg="License database preloaded" duration_ms=3936 pid=1447 vm-test-run-nixbot-gitlab> gitlab # [ 63.421634] postgres[1982]: [1982] ERROR: relation "feature_gates" does not exist at character 523 vm-test-run-nixbot-gitlab> gitlab # [ 63.423874] postgres[1982]: [1982] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), vm-test-run-nixbot-gitlab> gitlab # [ 63.425745] postgres[1982]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, vm-test-run-nixbot-gitlab> gitlab # [ 63.428426] postgres[1982]: c.collname, col_description(a.attrelid, a.attnum) AS comment, vm-test-run-nixbot-gitlab> gitlab # [ 63.429641] postgres[1982]: attidentity AS identity, vm-test-run-nixbot-gitlab> gitlab # [ 63.430451] postgres[1982]: attgenerated as attgenerated vm-test-run-nixbot-gitlab> gitlab # [ 63.431296] postgres[1982]: FROM pg_attribute a vm-test-run-nixbot-gitlab> gitlab # [ 63.432043] postgres[1982]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum vm-test-run-nixbot-gitlab> gitlab # [ 63.433314] postgres[1982]: LEFT JOIN pg_type t ON a.atttypid = t.oid vm-test-run-nixbot-gitlab> gitlab # [ 63.434289] postgres[1982]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation vm-test-run-nixbot-gitlab> gitlab # [ 63.435782] postgres[1982]: WHERE a.attrelid = '"feature_gates"'::regclass vm-test-run-nixbot-gitlab> gitlab # [ 63.436862] postgres[1982]: AND a.attnum > 0 AND NOT a.attisdropped vm-test-run-nixbot-gitlab> gitlab # [ 63.437773] postgres[1982]: ORDER BY a.attnum vm-test-run-nixbot-gitlab> gitlab # [ 63.438436] postgres[1982]: vm-test-run-nixbot-gitlab> gitlab # [ 63.992312] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1457]: W, [2026-07-25T11:26:43.803067 #1457] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 70.539694] gitlab-workhorse[1369]: time="2026-07-25T11:26:50Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 73.103644] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[2074]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping vm-test-run-nixbot-gitlab> gitlab # [ 73.104192] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[2074]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping vm-test-run-nixbot-gitlab> gitlab # [ 80.547627] gitlab-workhorse[1369]: time="2026-07-25T11:27:00Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 90.550951] gitlab-workhorse[1369]: time="2026-07-25T11:27:10Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 100.553060] gitlab-workhorse[1369]: time="2026-07-25T11:27:20Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 100.876851] postgres[2176]: [2176] ERROR: relation "table_without_models" does not exist vm-test-run-nixbot-gitlab> gitlab # [ 100.880610] postgres[2176]: [2176] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') vm-test-run-nixbot-gitlab> gitlab # [ 110.562681] gitlab-workhorse[1369]: time="2026-07-25T11:27:30Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 120.570431] gitlab-workhorse[1369]: time="2026-07-25T11:27:40Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 130.582612] gitlab-workhorse[1369]: time="2026-07-25T11:27:50Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 138.256004] systemd[1]: Finished gitlab-db-config.service. vm-test-run-nixbot-gitlab> gitlab # [ 138.264234] systemd[1]: Started gitlab-sidekiq.service. vm-test-run-nixbot-gitlab> gitlab # [ 138.271286] systemd[1]: Starting gitlab.service... vm-test-run-nixbot-gitlab> gitlab # [ 138.867363] bundle[2615]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 138.873971] bundle[2615]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 138.880114] bundle[2615]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 140.590282] gitlab-workhorse[1369]: time="2026-07-25T11:28:00Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 144.845449] sidekiq-cluster[2614]: {"severity":"INFO","time":"2026-07-25T11:28:04.617Z","message":"Starting cluster with 1 processes"} vm-test-run-nixbot-gitlab> gitlab # [ 145.095935] bundle[2615]: {"timestamp":"2026-07-25T11:28:04.861Z","pid":2615,"message":"* Enabling systemd notification integration"} vm-test-run-nixbot-gitlab> gitlab # [ 145.100535] bundle[2615]: {"timestamp":"2026-07-25T11:28:04.911Z","pid":2615,"message":"Puma starting in cluster mode..."} vm-test-run-nixbot-gitlab> gitlab # [ 145.105222] bundle[2615]: {"timestamp":"2026-07-25T11:28:04.911Z","pid":2615,"message":"* Puma version: 7.2.0 (\"On The Corner\")"} vm-test-run-nixbot-gitlab> gitlab # [ 145.110106] bundle[2615]: {"timestamp":"2026-07-25T11:28:04.914Z","pid":2615,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} vm-test-run-nixbot-gitlab> gitlab # [ 145.116106] bundle[2615]: {"timestamp":"2026-07-25T11:28:04.914Z","pid":2615,"message":"* Min threads: 0"} vm-test-run-nixbot-gitlab> gitlab # [ 145.119782] bundle[2615]: {"timestamp":"2026-07-25T11:28:04.914Z","pid":2615,"message":"* Max threads: 4"} vm-test-run-nixbot-gitlab> gitlab # [ 145.123560] bundle[2615]: {"timestamp":"2026-07-25T11:28:04.914Z","pid":2615,"message":"* Environment: production"} vm-test-run-nixbot-gitlab> gitlab # [ 145.128077] bundle[2615]: {"timestamp":"2026-07-25T11:28:04.914Z","pid":2615,"message":"* Master PID: 2615"} vm-test-run-nixbot-gitlab> gitlab # [ 145.131714] bundle[2615]: {"timestamp":"2026-07-25T11:28:04.914Z","pid":2615,"message":"* Workers: 2"} vm-test-run-nixbot-gitlab> gitlab # [ 145.135512] bundle[2615]: {"timestamp":"2026-07-25T11:28:04.917Z","pid":2615,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} vm-test-run-nixbot-gitlab> gitlab # [ 145.140215] bundle[2615]: {"timestamp":"2026-07-25T11:28:04.917Z","pid":2615,"message":"* Preloading application"} vm-test-run-nixbot-gitlab> gitlab # [ 150.598272] gitlab-workhorse[1369]: time="2026-07-25T11:28:10Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 160.607212] gitlab-workhorse[1369]: time="2026-07-25T11:28:20Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 170.627092] gitlab-workhorse[1369]: time="2026-07-25T11:28:30Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 180.635284] gitlab-workhorse[1369]: time="2026-07-25T11:28:40Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 190.636563] gitlab-workhorse[1369]: time="2026-07-25T11:28:50Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 192.569489] bundle[2615]: W, [2026-07-25T11:28:52.381528 #2615] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 196.707001] sidekiq-cluster[2673]: 2026-07-25T11:28:56.518Z pid=2673 tid=tmh INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2673, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>446558208} vm-test-run-nixbot-gitlab> gitlab # [ 197.066127] sidekiq-cluster[2673]: W, [2026-07-25T11:28:56.879089 #2673] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 197.131473] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:28:56.944Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} vm-test-run-nixbot-gitlab> gitlab # [ 200.639733] gitlab-workhorse[1369]: time="2026-07-25T11:29:00Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 200.843327] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.655Z","message":"GitLab reliable fetch activated!"} vm-test-run-nixbot-gitlab> gitlab # [ 200.848536] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.655Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} vm-test-run-nixbot-gitlab> gitlab # [ 200.896567] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.709Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 200.904471] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.717Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 200.911344] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.724Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 200.918115] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.731Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 200.924068] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.737Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 200.928748] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.741Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 200.932906] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.746Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 200.936684] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.749Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 200.939974] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.753Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 200.943759] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.756Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 200.947626] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.760Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 200.952481] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.765Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 200.956150] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.769Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 200.959311] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.772Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 200.962555] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.775Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 200.966173] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.779Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 200.969424] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.782Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 200.980696] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.793Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 200.990040] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.802Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.000046] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.813Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.008879] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.822Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.017551] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.830Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.028967] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.841Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.033899] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.846Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.039245] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.852Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.042803] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.855Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.048716] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.861Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.053245] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.866Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.057297] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.870Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.061493] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.874Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.066501] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.878Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.070982] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.882Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.075927] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.887Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.080413] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.890Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.160932] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.973Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.168102] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.977Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.174837] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.980Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.181679] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.983Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.188171] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.988Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.194596] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.992Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.200406] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.994Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.206154] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.998Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.212486] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:00.999Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.218101] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.003Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.223561] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.005Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.229369] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.009Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.234958] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.011Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.239824] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.015Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.244343] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.017Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.248340] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.021Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.252157] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.023Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.255707] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.026Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.259260] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.030Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.262626] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.044Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.266380] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.049Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.269616] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.053Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.272634] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.055Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.275532] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.059Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.278712] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.062Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.281609] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.064Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.284405] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.067Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.286856] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.070Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.289325] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.073Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.291803] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.076Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.294441] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.080Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.296986] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.082Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.299365] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.087Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.301762] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.090Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.304116] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.092Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.306519] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.094Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.308967] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.096Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.311257] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.098Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.313563] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.101Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.315775] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.103Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.318288] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.105Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.320680] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.107Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.323003] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.109Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.325308] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.112Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.327777] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.114Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.330126] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.116Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.332292] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.118Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.334621] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.120Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.337118] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.122Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.339414] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.125Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.341766] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.127Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.344134] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.130Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.346332] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.132Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.348808] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.135Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.351069] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.137Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.353479] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.139Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.355748] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.142Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.358226] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.143Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.360636] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.146Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.362925] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.148Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.365467] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.150Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.367467] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.153Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.369960] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.155Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.372272] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.158Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.374531] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.160Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.376809] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.171Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.379057] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.173Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.381382] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.175Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.383664] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.179Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.385864] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.181Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.388057] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.184Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.390387] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.187Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.392870] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.189Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.394987] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.191Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.397354] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.194Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 201.399561] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:01.196Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 210.642629] gitlab-workhorse[1369]: time="2026-07-25T11:29:10Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 220.654291] gitlab-workhorse[1369]: time="2026-07-25T11:29:20Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 230.662297] gitlab-workhorse[1369]: time="2026-07-25T11:29:30Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 240.674135] gitlab-workhorse[1369]: time="2026-07-25T11:29:40Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 244.716755] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:44.529Z","message":"Booted Rails 7.2.3 application in production environment"} vm-test-run-nixbot-gitlab> gitlab # [ 244.718974] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:44.530Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} vm-test-run-nixbot-gitlab> gitlab # [ 244.721648] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:44.530Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} vm-test-run-nixbot-gitlab> gitlab # [ 244.723524] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:44.531Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} vm-test-run-nixbot-gitlab> gitlab # [ 244.725779] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:44.535Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} vm-test-run-nixbot-gitlab> gitlab # [ 247.509421] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:47.321Z","message":"Cleaning working queues"} vm-test-run-nixbot-gitlab> gitlab # [ 247.512960] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:47.325Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c4da67c7eed47b7eba5c3da5","created_at":"2026-07-25T11:27:56.752Z","trace_propagation_headers":{"sentry-trace":"4c577443fbc045aa891501d7747a69ad-a8031034e9d84794","baggage":"sentry-trace_id=4c577443fbc045aa891501d7747a69ad,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"03aaf09e9fbd82e7d2252b48c238f6ff","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-25T11:27:56.860Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c4da67c7eed47b7eba5c3da5: start","job_status":"start","queue_duration_s":110.465207,"scheduling_latency_s":110.465265} vm-test-run-nixbot-gitlab> gitlab # [ 247.837624] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:29:47.649Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c4da67c7eed47b7eba5c3da5","created_at":"2026-07-25T11:27:56.752Z","trace_propagation_headers":{"sentry-trace":"4c577443fbc045aa891501d7747a69ad-a8031034e9d84794","baggage":"sentry-trace_id=4c577443fbc045aa891501d7747a69ad,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"03aaf09e9fbd82e7d2252b48c238f6ff","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-25T11:27:56.860Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c4da67c7eed47b7eba5c3da5: done: 0.324306 sec","job_status":"done","queue_duration_s":110.465207,"scheduling_latency_s":110.465265,"redis_calls":29,"redis_duration_s":0.011551,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.006806,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002883,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001862,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.15244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.17300914,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.324306,"completed_at":"2026-07-25T11:29:47.649Z","load_balancing_strategy":"primary","db_duration_s":0.126909,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 248.727700] bundle[2615]: {"timestamp":"2026-07-25T11:29:48.538Z","pid":2615,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} vm-test-run-nixbot-gitlab> gitlab # [ 248.732758] bundle[2615]: {"timestamp":"2026-07-25T11:29:48.538Z","pid":2615,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} vm-test-run-nixbot-gitlab> gitlab # [ 248.733716] bundle[2615]: {"timestamp":"2026-07-25T11:29:48.539Z","pid":2615,"message":"! #\u003cThread:0x0000f84116c97048 /nix/store/09wikq0z4z0s8mkafl8f2b6yp0p78frd-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/09wikq0z4z0s8mkafl8f2b6yp0p78frd-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} vm-test-run-nixbot-gitlab> gitlab # [ 248.755002] bundle[2615]: {"timestamp":"2026-07-25T11:29:48.539Z","pid":2615,"message":"! #\u003cThread:0x0000f84116efcf18 /nix/store/q2bwpwxril8z19gq9b1nnv72drlwhhg8-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/q2bwpwxril8z19gq9b1nnv72drlwhhg8-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} vm-test-run-nixbot-gitlab> gitlab # [ 248.772394] bundle[2615]: {"timestamp":"2026-07-25T11:29:48.539Z","pid":2615,"message":"Use Ctrl-C to stop"} vm-test-run-nixbot-gitlab> gitlab # [ 248.776572] bundle[2615]: {"timestamp":"2026-07-25T11:29:48.540Z","pid":2615,"message":"Sending status to systemd every 1.0 sec"} vm-test-run-nixbot-gitlab> gitlab # [ 249.782967] bundle[2615]: {"timestamp":"2026-07-25T11:29:49.594Z","pid":2615,"message":"- Worker 1 (PID: 3602) booted in 0.97s, phase: 0"} vm-test-run-nixbot-gitlab> gitlab # [ 249.785129] bundle[2615]: {"timestamp":"2026-07-25T11:29:49.595Z","pid":2615,"message":"- Worker 0 (PID: 3598) booted in 1.0s, phase: 0"} vm-test-run-nixbot-gitlab> gitlab # [ 249.798074] systemd[1]: Started gitlab.service. vm-test-run-nixbot-gitlab> gitlab # [ 249.801013] systemd[1]: Reached target Common target for all GitLab services.. vm-test-run-nixbot-gitlab> gitlab # [ 249.810505] systemd[1]: Starting setup-gitlab.service... vm-test-run-nixbot-gitlab> gitlab # [ 249.848223] setup-gitlab-start[3635]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' vm-test-run-nixbot-gitlab> gitlab # [ 254.561879] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:29:54 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 4669 vm-test-run-nixbot-gitlab> gitlab # [ 254.571480] setup-gitlab-start[3667]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token vm-test-run-nixbot-gitlab> gitlab # [ 254.574418] setup-gitlab-start[3668]: ++ jq -r .access_token vm-test-run-nixbot-gitlab> gitlab # [ 256.386350] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:29:56 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 1781 vm-test-run-nixbot-gitlab> gitlab # [ 256.392808] setup-gitlab-start[3635]: + BEARER=716aee96aeecb4756eb506a74f49c247de01a387adbb9a1e7559a28c762259ac vm-test-run-nixbot-gitlab> gitlab # [ 256.394150] setup-gitlab-start[3635]: + AUTH='Authorization: Bearer 716aee96aeecb4756eb506a74f49c247de01a387adbb9a1e7559a28c762259ac' vm-test-run-nixbot-gitlab> gitlab # [ 256.397145] setup-gitlab-start[3635]: + curl -fs -X PUT -H 'Authorization: Bearer 716aee96aeecb4756eb506a74f49c247de01a387adbb9a1e7559a28c762259ac' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true' vm-test-run-nixbot-gitlab> gitlab # [ 257.541085] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:29:57 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 1111 vm-test-run-nixbot-gitlab> gitlab # [ 257.547421] setup-gitlab-start[3635]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear vm-test-run-nixbot-gitlab> gitlab # [ 257.579257] runuser[3690]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) vm-test-run-nixbot-gitlab> gitlab # [ 258.549688] setup-gitlab-start[3691]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 258.556796] setup-gitlab-start[3691]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 258.563378] setup-gitlab-start[3691]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 259.510883] setup-gitlab-start[3691]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 259.511714] setup-gitlab-start[3691]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 259.524197] setup-gitlab-start[3691]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 261.425393] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:01.236Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"21d4ce7fa465eebb5ca742df","created_at":"2026-07-25T11:30:01.184Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c33579ba4995ee0ab36954d5a1ddd62e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-25T11:30:01.232Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-21d4ce7fa465eebb5ca742df: start","job_status":"start","queue_duration_s":0.003979,"scheduling_latency_s":0.004066} vm-test-run-nixbot-gitlab> gitlab # [ 261.711205] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:01.522Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"21d4ce7fa465eebb5ca742df","created_at":"2026-07-25T11:30:01.184Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c33579ba4995ee0ab36954d5a1ddd62e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-25T11:30:01.232Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-21d4ce7fa465eebb5ca742df: done: 0.28649 sec","job_status":"done","queue_duration_s":0.003979,"scheduling_latency_s":0.004066,"redis_calls":11,"redis_duration_s":0.042314000000000004,"redis_read_bytes":137,"redis_write_bytes":1227,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.004772,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":397,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.035442,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0021,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.089946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00186342,"duration_s":0.28649,"completed_at":"2026-07-25T11:30:01.522Z","load_balancing_strategy":"primary","db_duration_s":0.058044,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 261.800588] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:01.535Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"bb46e9fc0dd83572344012d4","created_at":"2026-07-25T11:30:01.246Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"475b57826269cebed235493139b0d1b4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.335Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-bb46e9fc0dd83572344012d4: start","job_status":"start","queue_duration_s":0.199623,"scheduling_latency_s":0.199687} vm-test-run-nixbot-gitlab> gitlab # [ 261.920721] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:01.732Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"bb46e9fc0dd83572344012d4","created_at":"2026-07-25T11:30:01.246Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"475b57826269cebed235493139b0d1b4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.335Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-bb46e9fc0dd83572344012d4: done: 0.19769 sec","job_status":"done","queue_duration_s":0.199623,"scheduling_latency_s":0.199687,"redis_calls":19,"redis_duration_s":0.05823,"redis_read_bytes":22,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.006036,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.001414,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.04672,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00406,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.083793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00350786,"duration_s":0.19769,"completed_at":"2026-07-25T11:30:01.732Z","load_balancing_strategy":"primary","db_duration_s":0.015669,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 262.003120] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:01.742Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"46d6442fa5533f60c0eeed6d","created_at":"2026-07-25T11:30:01.357Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d91063562abd0b7f899d95d07d89044d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.375Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-46d6442fa5533f60c0eeed6d: start","job_status":"start","queue_duration_s":0.367464,"scheduling_latency_s":0.36752} vm-test-run-nixbot-gitlab> gitlab # [ 262.921088] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:02.732Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"46d6442fa5533f60c0eeed6d","created_at":"2026-07-25T11:30:01.357Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d91063562abd0b7f899d95d07d89044d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.375Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-46d6442fa5533f60c0eeed6d: done: 0.989427 sec","job_status":"done","queue_duration_s":0.367464,"scheduling_latency_s":0.36752,"redis_calls":14,"redis_duration_s":0.024146,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.007299,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.013408,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003439,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.025,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.484012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00199078,"exclusive_lock_hold_duration_s":0.89357582,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.989427,"completed_at":"2026-07-25T11:30:02.732Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.044629,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 262.921634] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:02.734Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"d326eb511d0a6f2f42df1765","created_at":"2026-07-25T11:30:01.384Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fee293b4091c32360febea9916e5589c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.410Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-d326eb511d0a6f2f42df1765: start","job_status":"start","queue_duration_s":1.323943,"scheduling_latency_s":1.323973} vm-test-run-nixbot-gitlab> gitlab # [ 263.021933] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:02.764Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"d326eb511d0a6f2f42df1765","created_at":"2026-07-25T11:30:01.384Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fee293b4091c32360febea9916e5589c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.410Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-d326eb511d0a6f2f42df1765: done: 0.029794 sec","job_status":"done","queue_duration_s":1.323943,"scheduling_latency_s":1.323973,"redis_calls":12,"redis_duration_s":0.002251,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000795,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001031,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000425,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028852,"duration_s":0.029794,"completed_at":"2026-07-25T11:30:02.764Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001711,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 263.657650] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:02.766Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"766a4f5fcd6039331e7a88ba","created_at":"2026-07-25T11:30:01.443Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"94dd2c35e9b7fcc96461c9ebec2da6ca","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-25T11:30:01.472Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-766a4f5fcd6039331e7a88ba: start","job_status":"start","queue_duration_s":1.293496,"scheduling_latency_s":1.293521} vm-test-run-nixbot-gitlab> gitlab # [ 263.657766] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:02.780Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 263.657931] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:02.790Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"766a4f5fcd6039331e7a88ba","created_at":"2026-07-25T11:30:01.443Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"94dd2c35e9b7fcc96461c9ebec2da6ca","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-25T11:30:01.472Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-766a4f5fcd6039331e7a88ba: done: 0.024039 sec","job_status":"done","queue_duration_s":1.293496,"scheduling_latency_s":1.293521,"redis_calls":10,"redis_duration_s":0.00186,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000424,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001033,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000403,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032652,"duration_s":0.024039,"completed_at":"2026-07-25T11:30:02.789Z","load_balancing_strategy":"primary","db_duration_s":0.000638,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 264.379769] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:02.791Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","jid":"a430313305bcd5d1607c4768","created_at":"2026-07-25T11:30:01.504Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d2c9bafdf9bcd4f136e84fc8efcac0f6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fd978bae1387dafc89025ea66dbcf41624e32b5e06562317011e023b1327155","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.527Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleBulkDeleteJobArtifactCronWorker JID-a430313305bcd5d1607c4768: start","job_status":"start","queue_duration_s":1.264387,"scheduling_latency_s":1.264413} vm-test-run-nixbot-gitlab> gitlab # [ 264.423103] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:02.832Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","jid":"a430313305bcd5d1607c4768","created_at":"2026-07-25T11:30:01.504Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d2c9bafdf9bcd4f136e84fc8efcac0f6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fd978bae1387dafc89025ea66dbcf41624e32b5e06562317011e023b1327155","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.527Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleBulkDeleteJobArtifactCronWorker JID-a430313305bcd5d1607c4768: done: 0.040921 sec","job_status":"done","queue_duration_s":1.264387,"scheduling_latency_s":1.264413,"redis_calls":24,"redis_duration_s":0.003558,"redis_read_bytes":25,"redis_write_bytes":5977,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000678,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":704,"redis_queues_calls":2,"redis_queues_duration_s":0.000304,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3810,"redis_queues_metadata_calls":11,"redis_queues_metadata_duration_s":0.001621,"redis_queues_metadata_read_bytes":14,"redis_queues_metadata_write_bytes":894,"redis_shared_state_calls":7,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":569,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024222,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.max_buckets":5,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.available_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.occupied_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.stale_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.missing_count":5,"duration_s":0.040921,"completed_at":"2026-07-25T11:30:02.832Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001097,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 264.472341] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:02.833Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckExportJobsWorker","jid":"fc306bb08bdb9d9d90048c7f","created_at":"2026-07-25T11:30:01.544Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1006755b92a31417a2cf62517db01ec8","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:c05136a1912f23c3950f9f6243170d655fcc26d1e5b96896d7e248d72f9039fe","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.599Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckExportJobsWorker JID-fc306bb08bdb9d9d90048c7f: start","job_status":"start","queue_duration_s":1.234503,"scheduling_latency_s":1.234524} vm-test-run-nixbot-gitlab> gitlab # [ 264.486731] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:02.863Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckExportJobsWorker","jid":"fc306bb08bdb9d9d90048c7f","created_at":"2026-07-25T11:30:01.544Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1006755b92a31417a2cf62517db01ec8","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:c05136a1912f23c3950f9f6243170d655fcc26d1e5b96896d7e248d72f9039fe","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.599Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckExportJobsWorker JID-fc306bb08bdb9d9d90048c7f: done: 0.029823 sec","job_status":"done","queue_duration_s":1.234503,"scheduling_latency_s":1.234524,"redis_calls":10,"redis_duration_s":0.002381,"redis_read_bytes":9,"redis_write_bytes":995,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000423,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001385,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":518,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000573,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":192,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038032,"duration_s":0.029823,"completed_at":"2026-07-25T11:30:02.863Z","load_balancing_strategy":"primary","db_duration_s":0.005854,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 264.528837] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:02.866Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"4e1123541438943a8edf8b57","created_at":"2026-07-25T11:30:01.619Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8c34c7b40bf376d3411af7954dbf9c6c","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.655Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-4e1123541438943a8edf8b57: start","job_status":"start","queue_duration_s":1.210172,"scheduling_latency_s":1.210193} vm-test-run-nixbot-gitlab> gitlab # [ 264.543619] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:02.908Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"4e1123541438943a8edf8b57","created_at":"2026-07-25T11:30:01.619Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8c34c7b40bf376d3411af7954dbf9c6c","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.655Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-4e1123541438943a8edf8b57: done: 0.041931 sec","job_status":"done","queue_duration_s":1.210172,"scheduling_latency_s":1.210193,"redis_calls":12,"redis_duration_s":0.004424,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00049,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001881,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002053,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006125,"exclusive_lock_hold_duration_s":0.01027306,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.041931,"completed_at":"2026-07-25T11:30:02.907Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006444,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 264.588384] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:02.912Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"332487ad21688b74d3635019","created_at":"2026-07-25T11:30:01.604Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-1dc0342273eb4923","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"475b57826269cebed235493139b0d1b4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.667Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-332487ad21688b74d3635019: start","job_status":"start","queue_duration_s":1.244362,"scheduling_latency_s":1.244464} vm-test-run-nixbot-gitlab> gitlab # [ 264.600955] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:02.964Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 264.602669] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:02.999Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"332487ad21688b74d3635019","created_at":"2026-07-25T11:30:01.604Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-1dc0342273eb4923","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"475b57826269cebed235493139b0d1b4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.667Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-332487ad21688b74d3635019: done: 0.087941 sec","job_status":"done","queue_duration_s":1.244362,"scheduling_latency_s":1.244464,"redis_calls":13,"redis_duration_s":0.0077800000000000005,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002579,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00202,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003181,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057661,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00074864,"duration_s":0.087941,"completed_at":"2026-07-25T11:30:02.999Z","load_balancing_strategy":"primary","db_duration_s":0.010453,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 264.643877] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.002Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9fa7744bff8b8694d57b7ea9","created_at":"2026-07-25T11:30:01.604Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-1dc0342273eb4923","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"475b57826269cebed235493139b0d1b4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.667Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9fa7744bff8b8694d57b7ea9: start","job_status":"start","queue_duration_s":1.33486,"scheduling_latency_s":1.3349} vm-test-run-nixbot-gitlab> gitlab # [ 264.656683] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.022Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 264.658374] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.036Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9fa7744bff8b8694d57b7ea9","created_at":"2026-07-25T11:30:01.604Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-1dc0342273eb4923","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"475b57826269cebed235493139b0d1b4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.667Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9fa7744bff8b8694d57b7ea9: done: 0.034012 sec","job_status":"done","queue_duration_s":1.33486,"scheduling_latency_s":1.3349,"redis_calls":10,"redis_duration_s":0.003424,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001232,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002192,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034012,"completed_at":"2026-07-25T11:30:03.036Z","load_balancing_strategy":"primary","db_duration_s":0.002796,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 264.699360] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.038Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"033ba67e4886897c82bc9d8c","created_at":"2026-07-25T11:30:01.604Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-1dc0342273eb4923","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"475b57826269cebed235493139b0d1b4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.667Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-033ba67e4886897c82bc9d8c: start","job_status":"start","queue_duration_s":1.370825,"scheduling_latency_s":1.37086} vm-test-run-nixbot-gitlab> gitlab # [ 264.712132] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.055Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 264.713852] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.067Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"033ba67e4886897c82bc9d8c","created_at":"2026-07-25T11:30:01.604Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-1dc0342273eb4923","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"475b57826269cebed235493139b0d1b4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.667Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-033ba67e4886897c82bc9d8c: done: 0.02943 sec","job_status":"done","queue_duration_s":1.370825,"scheduling_latency_s":1.37086,"redis_calls":10,"redis_duration_s":0.003266,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001413,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001853,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019509,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02943,"completed_at":"2026-07-25T11:30:03.067Z","load_balancing_strategy":"primary","db_duration_s":0.002519,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 264.753368] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.069Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c17b5a1b1d9e7549b34398d8","created_at":"2026-07-25T11:30:01.604Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-1dc0342273eb4923","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"475b57826269cebed235493139b0d1b4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.667Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c17b5a1b1d9e7549b34398d8: start","job_status":"start","queue_duration_s":1.402048,"scheduling_latency_s":1.402079} vm-test-run-nixbot-gitlab> gitlab # [ 264.765673] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.084Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 264.767374] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.096Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c17b5a1b1d9e7549b34398d8","created_at":"2026-07-25T11:30:01.604Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-1dc0342273eb4923","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"475b57826269cebed235493139b0d1b4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.667Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c17b5a1b1d9e7549b34398d8: done: 0.027263 sec","job_status":"done","queue_duration_s":1.402048,"scheduling_latency_s":1.402079,"redis_calls":10,"redis_duration_s":0.002865,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001214,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001651,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018154,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027263,"completed_at":"2026-07-25T11:30:03.096Z","load_balancing_strategy":"primary","db_duration_s":0.002453,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 264.807258] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.098Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"717387886a5d4d2d1114aa1e","created_at":"2026-07-25T11:30:01.710Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c6615505159f4a1b0e9e6b3ce2fc1cdd","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.746Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-717387886a5d4d2d1114aa1e: start","job_status":"start","queue_duration_s":1.352093,"scheduling_latency_s":1.352123} vm-test-run-nixbot-gitlab> gitlab # [ 264.821152] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.146Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"717387886a5d4d2d1114aa1e","created_at":"2026-07-25T11:30:01.710Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c6615505159f4a1b0e9e6b3ce2fc1cdd","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.746Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-717387886a5d4d2d1114aa1e: done: 0.047682 sec","job_status":"done","queue_duration_s":1.352093,"scheduling_latency_s":1.352123,"redis_calls":10,"redis_duration_s":0.0028580000000000003,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000507,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001734,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000617,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005443,"duration_s":0.047682,"completed_at":"2026-07-25T11:30:03.146Z","load_balancing_strategy":"primary","db_duration_s":0.011408,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 264.863531] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.148Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"c76551fc59dd1a78ac89fdeb","created_at":"2026-07-25T11:30:01.783Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"983b85e9119b7449d7c380b224b5da1b","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.822Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-c76551fc59dd1a78ac89fdeb: start","job_status":"start","queue_duration_s":1.325819,"scheduling_latency_s":1.325847} vm-test-run-nixbot-gitlab> gitlab # [ 264.878575] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.195Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"c76551fc59dd1a78ac89fdeb","created_at":"2026-07-25T11:30:01.783Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"983b85e9119b7449d7c380b224b5da1b","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.822Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-c76551fc59dd1a78ac89fdeb: done: 0.047462 sec","job_status":"done","queue_duration_s":1.325819,"scheduling_latency_s":1.325847,"redis_calls":10,"redis_duration_s":0.003207,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000536,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001917,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000754,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00069736,"duration_s":0.047462,"completed_at":"2026-07-25T11:30:03.195Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006258,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 264.923196] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.199Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"8e62a45a90e0b49f8b306434","created_at":"2026-07-25T11:30:01.881Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe297e4cc1c2d922610508d2bf75e42b","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.904Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-8e62a45a90e0b49f8b306434: start","job_status":"start","queue_duration_s":1.294247,"scheduling_latency_s":1.294301} vm-test-run-nixbot-gitlab> gitlab # [ 264.937655] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.287Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"8e62a45a90e0b49f8b306434","created_at":"2026-07-25T11:30:01.881Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe297e4cc1c2d922610508d2bf75e42b","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.904Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-8e62a45a90e0b49f8b306434: done: 0.0886 sec","job_status":"done","queue_duration_s":1.294247,"scheduling_latency_s":1.294301,"redis_calls":10,"redis_duration_s":0.004576,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000873,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002749,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000954,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042583,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00108172,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.0886,"completed_at":"2026-07-25T11:30:03.287Z","load_balancing_strategy":"primary","db_duration_s":0.028587,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 264.981481] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.292Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"e8fff21e7b089ef8146a4664","created_at":"2026-07-25T11:30:01.920Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1b6b31028ac720a8ae68b71481c1661","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.937Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-e8fff21e7b089ef8146a4664: start","job_status":"start","queue_duration_s":1.354207,"scheduling_latency_s":1.354266} vm-test-run-nixbot-gitlab> gitlab # [ 264.995967] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.675Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"e8fff21e7b089ef8146a4664","created_at":"2026-07-25T11:30:01.920Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1b6b31028ac720a8ae68b71481c1661","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.937Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-e8fff21e7b089ef8146a4664: done: 0.38407 sec","job_status":"done","queue_duration_s":1.354207,"scheduling_latency_s":1.354266,"redis_calls":14,"redis_duration_s":0.0052770000000000004,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001594,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001936,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001747,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.064,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":125,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.064,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.300247,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00136264,"exclusive_lock_hold_duration_s":0.33246964,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.38407,"completed_at":"2026-07-25T11:30:03.675Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.093643,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 265.041673] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.677Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"a27a7aeca9ceb53249435940","created_at":"2026-07-25T11:30:01.958Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2af0f76b9dc4c1eb36ed443bb3a61cfd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.969Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-a27a7aeca9ceb53249435940: start","job_status":"start","queue_duration_s":1.707982,"scheduling_latency_s":1.708012} vm-test-run-nixbot-gitlab> gitlab # [ 265.056838] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.746Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"a27a7aeca9ceb53249435940","created_at":"2026-07-25T11:30:01.958Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2af0f76b9dc4c1eb36ed443bb3a61cfd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.969Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-a27a7aeca9ceb53249435940: done: 0.06823 sec","job_status":"done","queue_duration_s":1.707982,"scheduling_latency_s":1.708012,"redis_calls":12,"redis_duration_s":0.003869,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000598,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001775,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001496,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00097148,"exclusive_lock_hold_duration_s":0.022968719999999998,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.06823,"completed_at":"2026-07-25T11:30:03.746Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030259,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 265.103951] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.748Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"93ac10b0466badb4b1dcb6bc","created_at":"2026-07-25T11:30:01.977Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e4d988b8f5a06ad756362cf9d6017a4d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.988Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-93ac10b0466badb4b1dcb6bc: start","job_status":"start","queue_duration_s":1.760195,"scheduling_latency_s":1.760221} vm-test-run-nixbot-gitlab> gitlab # [ 265.118993] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.763Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 265.122379] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.771Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"93ac10b0466badb4b1dcb6bc","created_at":"2026-07-25T11:30:01.977Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e4d988b8f5a06ad756362cf9d6017a4d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:01.988Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-93ac10b0466badb4b1dcb6bc: done: 0.023406 sec","job_status":"done","queue_duration_s":1.760195,"scheduling_latency_s":1.760221,"redis_calls":10,"redis_duration_s":0.00245,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000487,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001459,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000504,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003404,"duration_s":0.023406,"completed_at":"2026-07-25T11:30:03.771Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000921,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 265.165578] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.774Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"4bfbe6ff387b69015aee4c37","created_at":"2026-07-25T11:30:01.996Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"54702a2233b850e69cd6ef4e1c9adfe6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.006Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-4bfbe6ff387b69015aee4c37: start","job_status":"start","queue_duration_s":1.767761,"scheduling_latency_s":1.767781} vm-test-run-nixbot-gitlab> gitlab # [ 265.180309] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.813Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"4bfbe6ff387b69015aee4c37","created_at":"2026-07-25T11:30:01.996Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"54702a2233b850e69cd6ef4e1c9adfe6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.006Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-4bfbe6ff387b69015aee4c37: done: 0.0391 sec","job_status":"done","queue_duration_s":1.767761,"scheduling_latency_s":1.767781,"redis_calls":20,"redis_duration_s":0.004709,"redis_read_bytes":23,"redis_write_bytes":4314,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001504,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1065,"redis_queues_calls":4,"redis_queues_duration_s":0.000741,"redis_queues_read_bytes":6,"redis_queues_write_bytes":2019,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001962,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":1004,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000502,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033302,"duration_s":0.0391,"completed_at":"2026-07-25T11:30:03.813Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001648,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 265.225447] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.814Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"6448c87441d633d33c609b5b","created_at":"2026-07-25T11:30:02.024Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"183d8f40f8f4e428df5589fb8ba88147","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.032Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-6448c87441d633d33c609b5b: start","job_status":"start","queue_duration_s":1.781805,"scheduling_latency_s":1.781823} vm-test-run-nixbot-gitlab> gitlab # [ 265.240098] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.846Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"6448c87441d633d33c609b5b","created_at":"2026-07-25T11:30:02.024Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"183d8f40f8f4e428df5589fb8ba88147","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.032Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-6448c87441d633d33c609b5b: done: 0.031722 sec","job_status":"done","queue_duration_s":1.781805,"scheduling_latency_s":1.781823,"redis_calls":17,"redis_duration_s":0.004169,"redis_read_bytes":15,"redis_write_bytes":2007,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001871,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1150,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001409,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000383,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000506,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020508,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00051672,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.031722,"completed_at":"2026-07-25T11:30:03.846Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001926,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 265.285541] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.848Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a38e78d0b2b8e930ef92508a","created_at":"2026-07-25T11:30:02.041Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"08bb982707f888b0509db6f0d07efc59","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.049Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a38e78d0b2b8e930ef92508a: start","job_status":"start","queue_duration_s":1.798173,"scheduling_latency_s":1.798192} vm-test-run-nixbot-gitlab> gitlab # [ 265.301450] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.861Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 265.305031] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.869Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a38e78d0b2b8e930ef92508a","created_at":"2026-07-25T11:30:02.041Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"08bb982707f888b0509db6f0d07efc59","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.049Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a38e78d0b2b8e930ef92508a: done: 0.02145 sec","job_status":"done","queue_duration_s":1.798173,"scheduling_latency_s":1.798192,"redis_calls":10,"redis_duration_s":0.002606,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000649,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00145,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000507,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011671,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032522,"duration_s":0.02145,"completed_at":"2026-07-25T11:30:03.869Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000493,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 265.348170] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.870Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"486cdcbe3c066cac02428316","created_at":"2026-07-25T11:30:02.058Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7fb71dd598a6dbf84b2f9fa0d2a89301","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.081Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-486cdcbe3c066cac02428316: start","job_status":"start","queue_duration_s":1.78946,"scheduling_latency_s":1.789477} vm-test-run-nixbot-gitlab> gitlab # [ 265.362869] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.891Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"486cdcbe3c066cac02428316","created_at":"2026-07-25T11:30:02.058Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7fb71dd598a6dbf84b2f9fa0d2a89301","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.081Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-486cdcbe3c066cac02428316: done: 0.020302 sec","job_status":"done","queue_duration_s":1.78946,"scheduling_latency_s":1.789477,"redis_calls":11,"redis_duration_s":0.00254,"redis_read_bytes":9,"redis_write_bytes":1239,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000457,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001372,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000711,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":298,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031484,"duration_s":0.020302,"completed_at":"2026-07-25T11:30:03.891Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000464,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 265.406860] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.892Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"0b3d4b844c8d1d4dfd606f3e","created_at":"2026-07-25T11:30:02.086Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93eb6e59853e18eef7760a8e01bd6da0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.095Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-0b3d4b844c8d1d4dfd606f3e: start","job_status":"start","queue_duration_s":1.797075,"scheduling_latency_s":1.797093} vm-test-run-nixbot-gitlab> gitlab # [ 265.422040] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.905Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 265.425995] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.913Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"0b3d4b844c8d1d4dfd606f3e","created_at":"2026-07-25T11:30:02.086Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93eb6e59853e18eef7760a8e01bd6da0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.095Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-0b3d4b844c8d1d4dfd606f3e: done: 0.020395 sec","job_status":"done","queue_duration_s":1.797075,"scheduling_latency_s":1.797093,"redis_calls":10,"redis_duration_s":0.002641,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000474,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001603,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000564,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011501,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005534,"duration_s":0.020395,"completed_at":"2026-07-25T11:30:03.913Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000542,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 265.469634] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.914Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"075464d767b44df6eb274eca","created_at":"2026-07-25T11:30:02.102Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9fae85ad27709e65c9d1fbfc46abdeb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.110Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-075464d767b44df6eb274eca: start","job_status":"start","queue_duration_s":1.803897,"scheduling_latency_s":1.803916} vm-test-run-nixbot-gitlab> gitlab # [ 265.484417] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.935Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"075464d767b44df6eb274eca","created_at":"2026-07-25T11:30:02.102Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9fae85ad27709e65c9d1fbfc46abdeb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.110Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-075464d767b44df6eb274eca: done: 0.020631 sec","job_status":"done","queue_duration_s":1.803897,"scheduling_latency_s":1.803916,"redis_calls":10,"redis_duration_s":0.0023,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000453,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001355,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000492,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032938,"duration_s":0.020631,"completed_at":"2026-07-25T11:30:03.935Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000494,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 265.529130] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.937Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"92c48d5b87c40d2052c1a467","created_at":"2026-07-25T11:30:02.127Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e674f07462e529a801de993cb800efe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.136Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-92c48d5b87c40d2052c1a467: start","job_status":"start","queue_duration_s":1.801026,"scheduling_latency_s":1.801046} vm-test-run-nixbot-gitlab> gitlab # [ 265.544173] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.954Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 265.547593] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.962Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"92c48d5b87c40d2052c1a467","created_at":"2026-07-25T11:30:02.127Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e674f07462e529a801de993cb800efe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.136Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-92c48d5b87c40d2052c1a467: done: 0.025417 sec","job_status":"done","queue_duration_s":1.801026,"scheduling_latency_s":1.801046,"redis_calls":10,"redis_duration_s":0.005208,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000723,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003947,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000538,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034192,"duration_s":0.025417,"completed_at":"2026-07-25T11:30:03.962Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000825,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 265.591588] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.963Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"1fe1f54478c842d8e56e6ef4","created_at":"2026-07-25T11:30:02.142Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e022ae1a0309becdb62c58dc4e158667","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.150Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-1fe1f54478c842d8e56e6ef4: start","job_status":"start","queue_duration_s":1.813253,"scheduling_latency_s":1.813271} vm-test-run-nixbot-gitlab> gitlab # [ 265.605282] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.989Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"1fe1f54478c842d8e56e6ef4","created_at":"2026-07-25T11:30:02.142Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e022ae1a0309becdb62c58dc4e158667","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.150Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-1fe1f54478c842d8e56e6ef4: done: 0.025822 sec","job_status":"done","queue_duration_s":1.813253,"scheduling_latency_s":1.813271,"redis_calls":10,"redis_duration_s":0.002808,"redis_read_bytes":9,"redis_write_bytes":1087,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000619,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001402,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000787,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016281,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032438,"duration_s":0.025822,"completed_at":"2026-07-25T11:30:03.989Z","load_balancing_strategy":"primary","db_duration_s":0.000846,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 265.646897] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:03.991Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"e52eaa7e1d077ff6b6fcb943","created_at":"2026-07-25T11:30:02.155Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"109811df964e2df471e65bea092e215e","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.166Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-e52eaa7e1d077ff6b6fcb943: start","job_status":"start","queue_duration_s":1.824864,"scheduling_latency_s":1.824881} vm-test-run-nixbot-gitlab> gitlab # [ 265.661978] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.033Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"e52eaa7e1d077ff6b6fcb943","created_at":"2026-07-25T11:30:02.155Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"109811df964e2df471e65bea092e215e","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.166Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-e52eaa7e1d077ff6b6fcb943: done: 0.042788 sec","job_status":"done","queue_duration_s":1.824864,"scheduling_latency_s":1.824881,"redis_calls":13,"redis_duration_s":0.006929,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000728,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002214,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003987,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01506446,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.042788,"completed_at":"2026-07-25T11:30:04.033Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007511,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 265.707623] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.035Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"c1ef7c2d6f1ed7a7e2a8082c","created_at":"2026-07-25T11:30:02.171Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11e5768c323af7438073d2ae2811a168","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.177Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-c1ef7c2d6f1ed7a7e2a8082c: start","job_status":"start","queue_duration_s":1.858024,"scheduling_latency_s":1.858043} vm-test-run-nixbot-gitlab> gitlab # [ 265.722533] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.067Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"c1ef7c2d6f1ed7a7e2a8082c","created_at":"2026-07-25T11:30:02.171Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11e5768c323af7438073d2ae2811a168","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.177Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-c1ef7c2d6f1ed7a7e2a8082c: done: 0.031807 sec","job_status":"done","queue_duration_s":1.858024,"scheduling_latency_s":1.858043,"redis_calls":11,"redis_duration_s":0.0026939999999999998,"redis_read_bytes":10,"redis_write_bytes":1022,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000474,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":283,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001432,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000788,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":225,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033428,"duration_s":0.031807,"completed_at":"2026-07-25T11:30:04.067Z","load_balancing_strategy":"primary","db_duration_s":0.006137,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 265.764715] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.069Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"d3b5a514c44898c06bec8a2e","created_at":"2026-07-25T11:30:02.181Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"95cc75dc30d8ebc6004b39e5d628b56a","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.188Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-d3b5a514c44898c06bec8a2e: start","job_status":"start","queue_duration_s":1.88023,"scheduling_latency_s":1.880249} vm-test-run-nixbot-gitlab> gitlab # [ 265.779730] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.102Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"d3b5a514c44898c06bec8a2e","created_at":"2026-07-25T11:30:02.181Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"95cc75dc30d8ebc6004b39e5d628b56a","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.188Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-d3b5a514c44898c06bec8a2e: done: 0.033585 sec","job_status":"done","queue_duration_s":1.88023,"scheduling_latency_s":1.880249,"redis_calls":13,"redis_duration_s":0.004156,"redis_read_bytes":10,"redis_write_bytes":1343,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000622,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002062,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001472,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":368,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003844,"extra.packages_cleanup_delete_orphaned_dependencies_worker.last_processed_packages_dependency_id":0,"extra.packages_cleanup_delete_orphaned_dependencies_worker.deleted_rows_count":0,"duration_s":0.033585,"completed_at":"2026-07-25T11:30:04.102Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006036,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 265.825972] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.104Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"d4d7e652f57352de6e763a0e","created_at":"2026-07-25T11:30:02.201Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b24a58ab9cd523c441cfc0bedda2b219","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.208Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-d4d7e652f57352de6e763a0e: start","job_status":"start","queue_duration_s":1.895594,"scheduling_latency_s":1.895612} vm-test-run-nixbot-gitlab> gitlab # [ 265.840886] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.138Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"d4d7e652f57352de6e763a0e","created_at":"2026-07-25T11:30:02.201Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b24a58ab9cd523c441cfc0bedda2b219","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.208Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-d4d7e652f57352de6e763a0e: done: 0.034362 sec","job_status":"done","queue_duration_s":1.895594,"scheduling_latency_s":1.895612,"redis_calls":12,"redis_duration_s":0.003401,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000625,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001447,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001329,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015567,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00062506,"exclusive_lock_hold_duration_s":0.00696934,"duration_s":0.034362,"completed_at":"2026-07-25T11:30:04.138Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00473,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 265.884532] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.140Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"b12ec70baa70eb9a5cf4236e","created_at":"2026-07-25T11:30:02.215Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2c7ceade21e301dfce35b4e8a18eab6a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-25T11:30:02.221Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-b12ec70baa70eb9a5cf4236e: start","job_status":"start","queue_duration_s":1.919007,"scheduling_latency_s":1.919029} vm-test-run-nixbot-gitlab> gitlab # [ 265.897915] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.154Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 265.901394] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.163Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"b12ec70baa70eb9a5cf4236e","created_at":"2026-07-25T11:30:02.215Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2c7ceade21e301dfce35b4e8a18eab6a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-25T11:30:02.221Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-b12ec70baa70eb9a5cf4236e: done: 0.022616 sec","job_status":"done","queue_duration_s":1.919007,"scheduling_latency_s":1.919029,"redis_calls":10,"redis_duration_s":0.002528,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000567,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001433,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000528,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01276,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035614,"duration_s":0.022616,"completed_at":"2026-07-25T11:30:04.163Z","load_balancing_strategy":"primary","db_duration_s":0.000862,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 265.945128] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.165Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Export::PruneProjectExportJobsWorker","jid":"11f44676ac9a9dd60fe0b8fb","created_at":"2026-07-25T11:30:02.229Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"572f973fc8bcc331f7bce35d0588cc51","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b2f7045536fe38235224f4d98e515e25b0b0eea1aa1ae4b4ab9f99ab22c53b43","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.237Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Export::PruneProjectExportJobsWorker JID-11f44676ac9a9dd60fe0b8fb: start","job_status":"start","queue_duration_s":1.927841,"scheduling_latency_s":1.927863} vm-test-run-nixbot-gitlab> gitlab # [ 265.959719] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.201Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Export::PruneProjectExportJobsWorker","jid":"11f44676ac9a9dd60fe0b8fb","created_at":"2026-07-25T11:30:02.229Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"572f973fc8bcc331f7bce35d0588cc51","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b2f7045536fe38235224f4d98e515e25b0b0eea1aa1ae4b4ab9f99ab22c53b43","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.237Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Export::PruneProjectExportJobsWorker JID-11f44676ac9a9dd60fe0b8fb: done: 0.036436 sec","job_status":"done","queue_duration_s":1.927841,"scheduling_latency_s":1.927863,"redis_calls":10,"redis_duration_s":0.0030819999999999997,"redis_read_bytes":9,"redis_write_bytes":1173,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000619,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001831,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000632,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":236,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00045012,"duration_s":0.036436,"completed_at":"2026-07-25T11:30:04.201Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004714,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 266.002594] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.203Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"50b89ac2b21439451ab8ba7b","created_at":"2026-07-25T11:30:02.245Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"520396b5919c1bd274dfef9aba97c670","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.253Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-50b89ac2b21439451ab8ba7b: start","job_status":"start","queue_duration_s":1.950213,"scheduling_latency_s":1.950237} vm-test-run-nixbot-gitlab> gitlab # [ 266.016108] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.229Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"50b89ac2b21439451ab8ba7b","created_at":"2026-07-25T11:30:02.245Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"520396b5919c1bd274dfef9aba97c670","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.253Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-50b89ac2b21439451ab8ba7b: done: 0.026364 sec","job_status":"done","queue_duration_s":1.950213,"scheduling_latency_s":1.950237,"redis_calls":12,"redis_duration_s":0.003451,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000607,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000699,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001497,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000648,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034648,"duration_s":0.026364,"completed_at":"2026-07-25T11:30:04.229Z","load_balancing_strategy":"primary","db_duration_s":0.000807,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 266.058892] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.231Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"72657db3b67064fdb0e8d0ea","created_at":"2026-07-25T11:30:02.260Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1daff056c0f81dc623ff3e5d6b74b2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.268Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-72657db3b67064fdb0e8d0ea: start","job_status":"start","queue_duration_s":1.963231,"scheduling_latency_s":1.963249} vm-test-run-nixbot-gitlab> gitlab # [ 266.073242] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.558Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"72657db3b67064fdb0e8d0ea","created_at":"2026-07-25T11:30:02.260Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1daff056c0f81dc623ff3e5d6b74b2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.268Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-72657db3b67064fdb0e8d0ea: done: 0.327134 sec","job_status":"done","queue_duration_s":1.963231,"scheduling_latency_s":1.963249,"redis_calls":574,"redis_duration_s":0.07229300000000001,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000568,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001227,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.070498,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.288048,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002847,"duration_s":0.327134,"completed_at":"2026-07-25T11:30:04.558Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000712,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 266.115470] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.560Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"056dbaafb92642a1ece6ef87","created_at":"2026-07-25T11:30:02.273Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.280Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-056dbaafb92642a1ece6ef87: start","job_status":"start","queue_duration_s":2.279505,"scheduling_latency_s":2.27953} vm-test-run-nixbot-gitlab> gitlab # [ 266.130379] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.701Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"056dbaafb92642a1ece6ef87","created_at":"2026-07-25T11:30:02.273Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.280Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-056dbaafb92642a1ece6ef87: done: 0.141158 sec","job_status":"done","queue_duration_s":2.279505,"scheduling_latency_s":2.27953,"redis_calls":71,"redis_duration_s":0.020482,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002429,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001131,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.016206,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000716,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.120128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038962,"duration_s":0.141158,"completed_at":"2026-07-25T11:30:04.701Z","load_balancing_strategy":"primary","db_duration_s":0.002389,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 266.173723] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.703Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"873ed1eb66202ee26983c2cb","created_at":"2026-07-25T11:30:02.286Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f6b25962d0955156b576d2ef6c04a940","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.292Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-873ed1eb66202ee26983c2cb: start","job_status":"start","queue_duration_s":2.410743,"scheduling_latency_s":2.410769} vm-test-run-nixbot-gitlab> gitlab # [ 266.187179] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.747Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"873ed1eb66202ee26983c2cb","created_at":"2026-07-25T11:30:02.286Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f6b25962d0955156b576d2ef6c04a940","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.292Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-873ed1eb66202ee26983c2cb: done: 0.044208 sec","job_status":"done","queue_duration_s":2.410743,"scheduling_latency_s":2.410769,"redis_calls":10,"redis_duration_s":0.004073,"redis_read_bytes":9,"redis_write_bytes":1097,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000605,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":309,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002755,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000713,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":218,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00065738,"duration_s":0.044208,"completed_at":"2026-07-25T11:30:04.747Z","load_balancing_strategy":"primary","db_duration_s":0.008769,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 266.228774] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.749Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d96edc63d3f6c5d627e7ffff","created_at":"2026-07-25T11:30:02.297Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37e5201755b40537a555ceca5642ae1b","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.303Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d96edc63d3f6c5d627e7ffff: start","job_status":"start","queue_duration_s":2.446266,"scheduling_latency_s":2.446292} vm-test-run-nixbot-gitlab> gitlab # [ 266.242933] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.787Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d96edc63d3f6c5d627e7ffff","created_at":"2026-07-25T11:30:02.297Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37e5201755b40537a555ceca5642ae1b","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.303Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d96edc63d3f6c5d627e7ffff: done: 0.037798 sec","job_status":"done","queue_duration_s":2.446266,"scheduling_latency_s":2.446292,"redis_calls":10,"redis_duration_s":0.003518,"redis_read_bytes":9,"redis_write_bytes":1109,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000632,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001766,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00112,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000474,"duration_s":0.037798,"completed_at":"2026-07-25T11:30:04.787Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009275,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 266.285765] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.789Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"61236b6d9b6ca09d2a6c7db4","created_at":"2026-07-25T11:30:02.309Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"691986cad883a73c27c52d711da830cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.316Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-61236b6d9b6ca09d2a6c7db4: start","job_status":"start","queue_duration_s":2.473049,"scheduling_latency_s":2.473071} vm-test-run-nixbot-gitlab> gitlab # [ 266.300584] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.813Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"61236b6d9b6ca09d2a6c7db4","created_at":"2026-07-25T11:30:02.309Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"691986cad883a73c27c52d711da830cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.316Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-61236b6d9b6ca09d2a6c7db4: done: 0.023356 sec","job_status":"done","queue_duration_s":2.473049,"scheduling_latency_s":2.473071,"redis_calls":12,"redis_duration_s":0.003396,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000584,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001418,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001394,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003496,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.023356,"completed_at":"2026-07-25T11:30:04.813Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00081,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 266.347500] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.815Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"d567ac748801da43b5a19c5f","created_at":"2026-07-25T11:30:02.333Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af38debac95cc4cb8f582c1a642f2a17","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.339Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-d567ac748801da43b5a19c5f: start","job_status":"start","queue_duration_s":2.475175,"scheduling_latency_s":2.475195} vm-test-run-nixbot-gitlab> gitlab # [ 266.361175] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.881Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"d567ac748801da43b5a19c5f","created_at":"2026-07-25T11:30:02.333Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af38debac95cc4cb8f582c1a642f2a17","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.339Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-d567ac748801da43b5a19c5f: done: 0.066834 sec","job_status":"done","queue_duration_s":2.475175,"scheduling_latency_s":2.475195,"redis_calls":10,"redis_duration_s":0.003211,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000596,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00196,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000655,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00052706,"extra.pages_deactivate_expired_deployments_cron_worker.deactivate_expired_pages_deployments":{"deactivated_deployments":0,"duration":0.00414356},"duration_s":0.066834,"completed_at":"2026-07-25T11:30:04.881Z","load_balancing_strategy":"primary","db_duration_s":0.029111,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 266.404838] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.884Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"773430631d914a40e03cfeff","created_at":"2026-07-25T11:30:02.344Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e259596d854957398822f775dfb0719f","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.350Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-773430631d914a40e03cfeff: start","job_status":"start","queue_duration_s":2.533222,"scheduling_latency_s":2.533248} vm-test-run-nixbot-gitlab> gitlab # [ 266.418981] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.912Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"773430631d914a40e03cfeff","created_at":"2026-07-25T11:30:02.344Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e259596d854957398822f775dfb0719f","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.350Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-773430631d914a40e03cfeff: done: 0.028792 sec","job_status":"done","queue_duration_s":2.533222,"scheduling_latency_s":2.533248,"redis_calls":10,"redis_duration_s":0.002788,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000642,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00157,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000576,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014441,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034618,"duration_s":0.028792,"completed_at":"2026-07-25T11:30:04.912Z","load_balancing_strategy":"primary","db_duration_s":0.004279,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 266.461450] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.914Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"0462dec0c64afefcd055c693","created_at":"2026-07-25T11:30:02.355Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"541b9b2b10a18f7a2b64ca56dda69abd","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.362Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-0462dec0c64afefcd055c693: start","job_status":"start","queue_duration_s":2.552491,"scheduling_latency_s":2.552512} vm-test-run-nixbot-gitlab> gitlab # [ 266.475399] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.940Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"0462dec0c64afefcd055c693","created_at":"2026-07-25T11:30:02.355Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"541b9b2b10a18f7a2b64ca56dda69abd","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.362Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-0462dec0c64afefcd055c693: done: 0.025928 sec","job_status":"done","queue_duration_s":2.552491,"scheduling_latency_s":2.552512,"redis_calls":10,"redis_duration_s":0.002526,"redis_read_bytes":9,"redis_write_bytes":1179,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000579,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001429,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000518,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":238,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016369,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033262,"duration_s":0.025928,"completed_at":"2026-07-25T11:30:04.940Z","load_balancing_strategy":"primary","db_duration_s":0.000731,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 266.517622] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.942Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Import::ImportFileCleanupWorker","jid":"236cca1b737981a8e9107eca","created_at":"2026-07-25T11:30:02.366Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84b5b1eac4669696a3d522719f3eac57","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:872203d04eabf30611038084c617b8ff07fd3e5607b1d8e2c977d615a3cfd3cb","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.373Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Import::ImportFileCleanupWorker JID-236cca1b737981a8e9107eca: start","job_status":"start","queue_duration_s":2.568909,"scheduling_latency_s":2.568928} vm-test-run-nixbot-gitlab> gitlab # [ 266.533300] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.974Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Import::ImportFileCleanupWorker","jid":"236cca1b737981a8e9107eca","created_at":"2026-07-25T11:30:02.366Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84b5b1eac4669696a3d522719f3eac57","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:872203d04eabf30611038084c617b8ff07fd3e5607b1d8e2c977d615a3cfd3cb","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.373Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Import::ImportFileCleanupWorker JID-236cca1b737981a8e9107eca: done: 0.032224 sec","job_status":"done","queue_duration_s":2.568909,"scheduling_latency_s":2.568928,"redis_calls":10,"redis_duration_s":0.0028409999999999998,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000527,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0017,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000614,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004673,"duration_s":0.032224,"completed_at":"2026-07-25T11:30:04.974Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00647,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 266.576307] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:04.976Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c1d062adcc74338ec364775a","created_at":"2026-07-25T11:30:02.379Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c740c5786f2a4ce1c34a1cfac7c282e5","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.386Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c1d062adcc74338ec364775a: start","job_status":"start","queue_duration_s":2.589538,"scheduling_latency_s":2.58956} vm-test-run-nixbot-gitlab> gitlab # [ 266.590633] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.007Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c1d062adcc74338ec364775a","created_at":"2026-07-25T11:30:02.379Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c740c5786f2a4ce1c34a1cfac7c282e5","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.386Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c1d062adcc74338ec364775a: done: 0.031002 sec","job_status":"done","queue_duration_s":2.589538,"scheduling_latency_s":2.58956,"redis_calls":10,"redis_duration_s":0.003817,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000617,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002572,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000628,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00054412,"duration_s":0.031002,"completed_at":"2026-07-25T11:30:05.007Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00496,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 266.633557] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.009Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"b8f22f2d230b552266c6bfb6","created_at":"2026-07-25T11:30:02.410Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d2e1420fbb78f882c58f6661822ebe9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.416Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-b8f22f2d230b552266c6bfb6: start","job_status":"start","queue_duration_s":2.592102,"scheduling_latency_s":2.592123} vm-test-run-nixbot-gitlab> gitlab # [ 266.647278] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.051Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"b8f22f2d230b552266c6bfb6","created_at":"2026-07-25T11:30:02.410Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d2e1420fbb78f882c58f6661822ebe9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.416Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-b8f22f2d230b552266c6bfb6: done: 0.042596 sec","job_status":"done","queue_duration_s":2.592102,"scheduling_latency_s":2.592123,"redis_calls":12,"redis_duration_s":0.006972000000000001,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000816,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002023,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004133,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019457,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0032235,"exclusive_lock_hold_duration_s":0.00965664,"duration_s":0.042596,"completed_at":"2026-07-25T11:30:05.051Z","load_balancing_strategy":"primary","db_duration_s":0.006197,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 266.689448] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"9abac74eef3f1dd2410afdfe","created_at":"2026-07-25T11:30:02.807Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-82a9efa929244748","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"d2c9bafdf9bcd4f136e84fc8efcac0f6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.824Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-9abac74eef3f1dd2410afdfe: start","job_status":"start","queue_duration_s":2.229353,"scheduling_latency_s":2.229374} vm-test-run-nixbot-gitlab> gitlab # [ 266.702248] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.111Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"9abac74eef3f1dd2410afdfe","created_at":"2026-07-25T11:30:02.807Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-82a9efa929244748","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"d2c9bafdf9bcd4f136e84fc8efcac0f6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.824Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-9abac74eef3f1dd2410afdfe: done: 0.057696 sec","job_status":"done","queue_duration_s":2.229353,"scheduling_latency_s":2.229374,"redis_calls":18,"redis_duration_s":0.008321,"redis_read_bytes":39,"redis_write_bytes":2116,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002951,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001247,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":471,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.004123,"redis_shared_state_read_bytes":34,"redis_shared_state_write_bytes":1324,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025832,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00047884,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":3,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":3,"duration_s":0.057696,"completed_at":"2026-07-25T11:30:05.111Z","load_balancing_strategy":"primary","db_duration_s":0.01799,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 266.748102] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.113Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"a0bf52675f55b523a57c9534","created_at":"2026-07-25T11:30:02.807Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-82a9efa929244748","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"d2c9bafdf9bcd4f136e84fc8efcac0f6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.824Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-a0bf52675f55b523a57c9534: start","job_status":"start","queue_duration_s":2.288889,"scheduling_latency_s":2.288911} vm-test-run-nixbot-gitlab> gitlab # [ 266.760785] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"a0bf52675f55b523a57c9534","created_at":"2026-07-25T11:30:02.807Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-82a9efa929244748","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"d2c9bafdf9bcd4f136e84fc8efcac0f6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.824Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-a0bf52675f55b523a57c9534: done: 0.028049 sec","job_status":"done","queue_duration_s":2.288889,"scheduling_latency_s":2.288911,"redis_calls":15,"redis_duration_s":0.0035700000000000003,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001113,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.002457,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":2,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":2,"duration_s":0.028049,"completed_at":"2026-07-25T11:30:05.141Z","load_balancing_strategy":"primary","db_duration_s":0.00448,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 266.803634] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"31586a251e8fd80104007334","created_at":"2026-07-25T11:30:02.807Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-82a9efa929244748","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"d2c9bafdf9bcd4f136e84fc8efcac0f6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.824Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-31586a251e8fd80104007334: start","job_status":"start","queue_duration_s":2.318457,"scheduling_latency_s":2.318477} vm-test-run-nixbot-gitlab> gitlab # [ 266.816444] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.170Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"31586a251e8fd80104007334","created_at":"2026-07-25T11:30:02.807Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-82a9efa929244748","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"d2c9bafdf9bcd4f136e84fc8efcac0f6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.824Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-31586a251e8fd80104007334: done: 0.027726 sec","job_status":"done","queue_duration_s":2.318457,"scheduling_latency_s":2.318477,"redis_calls":15,"redis_duration_s":0.003409,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001101,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.002308,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016234,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":3,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":3,"duration_s":0.027726,"completed_at":"2026-07-25T11:30:05.170Z","load_balancing_strategy":"primary","db_duration_s":0.004379,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 266.858708] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"7aa64e16e5ef1195356a7c86","created_at":"2026-07-25T11:30:02.807Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-82a9efa929244748","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"d2c9bafdf9bcd4f136e84fc8efcac0f6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.824Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-7aa64e16e5ef1195356a7c86: start","job_status":"start","queue_duration_s":2.34772,"scheduling_latency_s":2.347741} vm-test-run-nixbot-gitlab> gitlab # [ 266.871921] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"7aa64e16e5ef1195356a7c86","created_at":"2026-07-25T11:30:02.807Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-82a9efa929244748","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"d2c9bafdf9bcd4f136e84fc8efcac0f6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.824Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-7aa64e16e5ef1195356a7c86: done: 0.027291 sec","job_status":"done","queue_duration_s":2.34772,"scheduling_latency_s":2.347741,"redis_calls":15,"redis_duration_s":0.0034879999999999998,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.002431,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":0,"duration_s":0.027291,"completed_at":"2026-07-25T11:30:05.199Z","load_balancing_strategy":"primary","db_duration_s":0.004691,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 266.914728] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.200Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"f54daaf15f4b35b0c56e9953","created_at":"2026-07-25T11:30:02.807Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-82a9efa929244748","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"d2c9bafdf9bcd4f136e84fc8efcac0f6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.824Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-f54daaf15f4b35b0c56e9953: start","job_status":"start","queue_duration_s":2.376712,"scheduling_latency_s":2.376732} vm-test-run-nixbot-gitlab> gitlab # [ 266.927423] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.227Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"f54daaf15f4b35b0c56e9953","created_at":"2026-07-25T11:30:02.807Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-82a9efa929244748","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"d2c9bafdf9bcd4f136e84fc8efcac0f6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:02.824Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-f54daaf15f4b35b0c56e9953: done: 0.026189 sec","job_status":"done","queue_duration_s":2.376712,"scheduling_latency_s":2.376732,"redis_calls":15,"redis_duration_s":0.003392,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00109,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.002302,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":3,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":3,"duration_s":0.026189,"completed_at":"2026-07-25T11:30:05.227Z","load_balancing_strategy":"primary","db_duration_s":0.004513,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 266.970908] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.229Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"c7a9bbc1e9205df2c5d3597f","created_at":"2026-07-25T11:30:03.788Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-9745abc428b343c6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"54702a2233b850e69cd6ef4e1c9adfe6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:03.798Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-c7a9bbc1e9205df2c5d3597f: start","job_status":"start","queue_duration_s":1.430848,"scheduling_latency_s":1.430869} vm-test-run-nixbot-gitlab> gitlab # [ 266.986255] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.261Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"c7a9bbc1e9205df2c5d3597f","created_at":"2026-07-25T11:30:03.788Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-9745abc428b343c6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"54702a2233b850e69cd6ef4e1c9adfe6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:03.798Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-c7a9bbc1e9205df2c5d3597f: done: 0.032377 sec","job_status":"done","queue_duration_s":1.430848,"scheduling_latency_s":1.430869,"redis_calls":10,"redis_duration_s":0.002813,"redis_read_bytes":9,"redis_write_bytes":1117,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000579,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0017,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000534,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":222,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018649,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034534,"duration_s":0.032377,"completed_at":"2026-07-25T11:30:05.261Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005401,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 267.029667] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.263Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"a110d60aa8b3ca576cbefd10","created_at":"2026-07-25T11:30:03.799Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-9745abc428b343c6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"54702a2233b850e69cd6ef4e1c9adfe6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:03.804Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-a110d60aa8b3ca576cbefd10: start","job_status":"start","queue_duration_s":1.458265,"scheduling_latency_s":1.458286} vm-test-run-nixbot-gitlab> gitlab # [ 267.045152] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.289Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"a110d60aa8b3ca576cbefd10","created_at":"2026-07-25T11:30:03.799Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-9745abc428b343c6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"54702a2233b850e69cd6ef4e1c9adfe6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-07-25T11:30:03.804Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-a110d60aa8b3ca576cbefd10: done: 0.026538 sec","job_status":"done","queue_duration_s":1.458265,"scheduling_latency_s":1.458286,"redis_calls":10,"redis_duration_s":0.002707,"redis_read_bytes":9,"redis_write_bytes":1133,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000621,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001519,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000567,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034986,"duration_s":0.026538,"completed_at":"2026-07-25T11:30:05.289Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001479,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 267.088974] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.292Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52c8cea8cb0e39186d3af498","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52c8cea8cb0e39186d3af498: start","job_status":"start","queue_duration_s":0.602657,"scheduling_latency_s":0.602678} vm-test-run-nixbot-gitlab> gitlab # [ 267.101659] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52c8cea8cb0e39186d3af498","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52c8cea8cb0e39186d3af498: done: 0.02637 sec","job_status":"done","queue_duration_s":0.602657,"scheduling_latency_s":0.602678,"redis_calls":16,"redis_duration_s":0.003948,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000578,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001162,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002208,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035688,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02637,"completed_at":"2026-07-25T11:30:05.318Z","load_balancing_strategy":"primary","db_duration_s":0.000729,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 267.144147] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aff6e15212bd1b48953003a2","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aff6e15212bd1b48953003a2: start","job_status":"start","queue_duration_s":0.630795,"scheduling_latency_s":0.630814} vm-test-run-nixbot-gitlab> gitlab # [ 267.156774] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aff6e15212bd1b48953003a2","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aff6e15212bd1b48953003a2: done: 0.01838 sec","job_status":"done","queue_duration_s":0.630795,"scheduling_latency_s":0.630814,"redis_calls":13,"redis_duration_s":0.002761,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001778,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011575,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01838,"completed_at":"2026-07-25T11:30:05.338Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 267.196776] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.340Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52d17e78c0b9f6ff69e2f75d","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52d17e78c0b9f6ff69e2f75d: start","job_status":"start","queue_duration_s":0.65083,"scheduling_latency_s":0.650848} vm-test-run-nixbot-gitlab> gitlab # [ 267.209356] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.358Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52d17e78c0b9f6ff69e2f75d","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52d17e78c0b9f6ff69e2f75d: done: 0.018568 sec","job_status":"done","queue_duration_s":0.65083,"scheduling_latency_s":0.650848,"redis_calls":13,"redis_duration_s":0.0029360000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000995,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001941,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018568,"completed_at":"2026-07-25T11:30:05.358Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 267.249155] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.360Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"960c03e60b60ddfbeb02cb58","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-960c03e60b60ddfbeb02cb58: start","job_status":"start","queue_duration_s":0.671022,"scheduling_latency_s":0.671046} vm-test-run-nixbot-gitlab> gitlab # [ 267.261840] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"960c03e60b60ddfbeb02cb58","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-960c03e60b60ddfbeb02cb58: done: 0.016738 sec","job_status":"done","queue_duration_s":0.671022,"scheduling_latency_s":0.671046,"redis_calls":13,"redis_duration_s":0.002656,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000907,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001749,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016738,"completed_at":"2026-07-25T11:30:05.377Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 267.301904] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91a4c5a1b0091eed0d56c132","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91a4c5a1b0091eed0d56c132: start","job_status":"start","queue_duration_s":0.689421,"scheduling_latency_s":0.68944} vm-test-run-nixbot-gitlab> gitlab # [ 267.314625] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91a4c5a1b0091eed0d56c132","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91a4c5a1b0091eed0d56c132: done: 0.016702 sec","job_status":"done","queue_duration_s":0.689421,"scheduling_latency_s":0.68944,"redis_calls":13,"redis_duration_s":0.003264,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002343,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016702,"completed_at":"2026-07-25T11:30:05.395Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 267.356049] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc7378fa28880ed845ffb55a","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc7378fa28880ed845ffb55a: start","job_status":"start","queue_duration_s":0.707703,"scheduling_latency_s":0.70772} vm-test-run-nixbot-gitlab> gitlab # [ 267.368657] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.413Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc7378fa28880ed845ffb55a","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc7378fa28880ed845ffb55a: done: 0.016872 sec","job_status":"done","queue_duration_s":0.707703,"scheduling_latency_s":0.70772,"redis_calls":13,"redis_duration_s":0.00272,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001783,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016872,"completed_at":"2026-07-25T11:30:05.413Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 267.408750] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.415Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8984cef45d107319ec0acd45","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8984cef45d107319ec0acd45: start","job_status":"start","queue_duration_s":0.72599,"scheduling_latency_s":0.726009} vm-test-run-nixbot-gitlab> gitlab # [ 267.421823] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8984cef45d107319ec0acd45","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8984cef45d107319ec0acd45: done: 0.017344 sec","job_status":"done","queue_duration_s":0.72599,"scheduling_latency_s":0.726009,"redis_calls":13,"redis_duration_s":0.002773,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000953,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00182,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017344,"completed_at":"2026-07-25T11:30:05.432Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 267.461689] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.434Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ef2289065aa258e1ca64a7c","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ef2289065aa258e1ca64a7c: start","job_status":"start","queue_duration_s":0.744824,"scheduling_latency_s":0.744843} vm-test-run-nixbot-gitlab> gitlab # [ 267.474189] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.841Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ef2289065aa258e1ca64a7c","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ef2289065aa258e1ca64a7c: done: 0.40677 sec","job_status":"done","queue_duration_s":0.744824,"scheduling_latency_s":0.744843,"redis_calls":13,"redis_duration_s":0.360958,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.094049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.266909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.40677,"completed_at":"2026-07-25T11:30:05.840Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 267.513537] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.845Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dc82d943c4cdb526b0a8f0d","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dc82d943c4cdb526b0a8f0d: start","job_status":"start","queue_duration_s":1.155587,"scheduling_latency_s":1.155646} vm-test-run-nixbot-gitlab> gitlab # [ 267.526093] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dc82d943c4cdb526b0a8f0d","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dc82d943c4cdb526b0a8f0d: done: 0.043145 sec","job_status":"done","queue_duration_s":1.155587,"scheduling_latency_s":1.155646,"redis_calls":13,"redis_duration_s":0.00451,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00159,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00292,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029437,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.043145,"completed_at":"2026-07-25T11:30:05.887Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 267.566720] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.891Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e23c3d96073f45c05f0169a","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e23c3d96073f45c05f0169a: start","job_status":"start","queue_duration_s":1.201688,"scheduling_latency_s":1.20173} vm-test-run-nixbot-gitlab> gitlab # [ 267.579329] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e23c3d96073f45c05f0169a","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e23c3d96073f45c05f0169a: done: 0.029154 sec","job_status":"done","queue_duration_s":1.201688,"scheduling_latency_s":1.20173,"redis_calls":13,"redis_duration_s":0.0038339999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001143,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002691,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018641,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029154,"completed_at":"2026-07-25T11:30:05.920Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 267.618904] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad66d58b1e940367da136760","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad66d58b1e940367da136760: start","job_status":"start","queue_duration_s":1.232678,"scheduling_latency_s":1.232703} vm-test-run-nixbot-gitlab> gitlab # [ 267.631548] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad66d58b1e940367da136760","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad66d58b1e940367da136760: done: 0.023458 sec","job_status":"done","queue_duration_s":1.232678,"scheduling_latency_s":1.232703,"redis_calls":13,"redis_duration_s":0.0031669999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001027,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00214,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023458,"completed_at":"2026-07-25T11:30:05.945Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 267.671650] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c12535b72055ef03aafa4169","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c12535b72055ef03aafa4169: start","job_status":"start","queue_duration_s":1.257816,"scheduling_latency_s":1.257836} vm-test-run-nixbot-gitlab> gitlab # [ 267.684411] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.968Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c12535b72055ef03aafa4169","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c12535b72055ef03aafa4169: done: 0.021103 sec","job_status":"done","queue_duration_s":1.257816,"scheduling_latency_s":1.257836,"redis_calls":13,"redis_duration_s":0.003889,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00104,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002849,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021103,"completed_at":"2026-07-25T11:30:05.968Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 267.723681] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b575fef3dbdcd4f1fbc2e58b","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b575fef3dbdcd4f1fbc2e58b: start","job_status":"start","queue_duration_s":1.280606,"scheduling_latency_s":1.280627} vm-test-run-nixbot-gitlab> gitlab # [ 267.736360] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b575fef3dbdcd4f1fbc2e58b","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b575fef3dbdcd4f1fbc2e58b: done: 0.019897 sec","job_status":"done","queue_duration_s":1.280606,"scheduling_latency_s":1.280627,"redis_calls":13,"redis_duration_s":0.00374,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001803,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019897,"completed_at":"2026-07-25T11:30:05.989Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 267.777183] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:05.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e3886878ecee475a1ba66e6","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e3886878ecee475a1ba66e6: start","job_status":"start","queue_duration_s":1.303114,"scheduling_latency_s":1.303134} vm-test-run-nixbot-gitlab> gitlab # [ 267.789745] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e3886878ecee475a1ba66e6","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e3886878ecee475a1ba66e6: done: 0.019062 sec","job_status":"done","queue_duration_s":1.303114,"scheduling_latency_s":1.303134,"redis_calls":13,"redis_duration_s":0.002899,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000992,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019062,"completed_at":"2026-07-25T11:30:06.011Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 267.829217] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"496b4d577d57492aefc4655d","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-496b4d577d57492aefc4655d: start","job_status":"start","queue_duration_s":1.32366,"scheduling_latency_s":1.323678} vm-test-run-nixbot-gitlab> gitlab # [ 267.841901] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.031Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"496b4d577d57492aefc4655d","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-496b4d577d57492aefc4655d: done: 0.018825 sec","job_status":"done","queue_duration_s":1.32366,"scheduling_latency_s":1.323678,"redis_calls":13,"redis_duration_s":0.002961,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000987,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012386,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018825,"completed_at":"2026-07-25T11:30:06.031Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 267.882575] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb82a33c5d7e5839e8744245","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb82a33c5d7e5839e8744245: start","job_status":"start","queue_duration_s":1.343955,"scheduling_latency_s":1.343999} vm-test-run-nixbot-gitlab> gitlab # [ 267.895575] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.049Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb82a33c5d7e5839e8744245","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb82a33c5d7e5839e8744245: done: 0.016508 sec","job_status":"done","queue_duration_s":1.343955,"scheduling_latency_s":1.343999,"redis_calls":13,"redis_duration_s":0.002663,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001746,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009843,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016508,"completed_at":"2026-07-25T11:30:06.049Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 267.936105] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.052Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92247066e34084cb24538700","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92247066e34084cb24538700: start","job_status":"start","queue_duration_s":1.362735,"scheduling_latency_s":1.362754} vm-test-run-nixbot-gitlab> gitlab # [ 267.948836] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92247066e34084cb24538700","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92247066e34084cb24538700: done: 0.017701 sec","job_status":"done","queue_duration_s":1.362735,"scheduling_latency_s":1.362754,"redis_calls":13,"redis_duration_s":0.002726,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001789,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011769,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017701,"completed_at":"2026-07-25T11:30:06.069Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 267.989297] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.071Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ef0eda7e08c4961f5135456","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ef0eda7e08c4961f5135456: start","job_status":"start","queue_duration_s":1.381958,"scheduling_latency_s":1.381976} vm-test-run-nixbot-gitlab> gitlab # [ 268.001775] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ef0eda7e08c4961f5135456","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ef0eda7e08c4961f5135456: done: 0.026558 sec","job_status":"done","queue_duration_s":1.381958,"scheduling_latency_s":1.381976,"redis_calls":13,"redis_duration_s":0.009743,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001764,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007979,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026558,"completed_at":"2026-07-25T11:30:06.097Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 268.041004] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.099Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fbc9fa7c25a7713e772a4832","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fbc9fa7c25a7713e772a4832: start","job_status":"start","queue_duration_s":1.409964,"scheduling_latency_s":1.409983} vm-test-run-nixbot-gitlab> gitlab # [ 268.053452] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fbc9fa7c25a7713e772a4832","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fbc9fa7c25a7713e772a4832: done: 0.018222 sec","job_status":"done","queue_duration_s":1.409964,"scheduling_latency_s":1.409983,"redis_calls":13,"redis_duration_s":0.002659,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000864,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001795,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018222,"completed_at":"2026-07-25T11:30:06.117Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 268.092651] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.121Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcd066cff50b85342ede8754","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcd066cff50b85342ede8754: start","job_status":"start","queue_duration_s":1.431876,"scheduling_latency_s":1.431896} vm-test-run-nixbot-gitlab> gitlab # [ 268.105092] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.144Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcd066cff50b85342ede8754","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcd066cff50b85342ede8754: done: 0.023295 sec","job_status":"done","queue_duration_s":1.431876,"scheduling_latency_s":1.431896,"redis_calls":13,"redis_duration_s":0.006175,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001661,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004514,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023295,"completed_at":"2026-07-25T11:30:06.144Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 268.144693] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34101a48cdabb81288c3e446","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34101a48cdabb81288c3e446: start","job_status":"start","queue_duration_s":1.456801,"scheduling_latency_s":1.456822} vm-test-run-nixbot-gitlab> gitlab # [ 268.158229] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34101a48cdabb81288c3e446","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34101a48cdabb81288c3e446: done: 0.01931 sec","job_status":"done","queue_duration_s":1.456801,"scheduling_latency_s":1.456822,"redis_calls":13,"redis_duration_s":0.00303,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00103,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01931,"completed_at":"2026-07-25T11:30:06.165Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 268.197242] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a13e55495dc46b083c87c28c","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a13e55495dc46b083c87c28c: start","job_status":"start","queue_duration_s":1.477863,"scheduling_latency_s":1.477883} vm-test-run-nixbot-gitlab> gitlab # [ 268.209671] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a13e55495dc46b083c87c28c","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a13e55495dc46b083c87c28c: done: 0.019519 sec","job_status":"done","queue_duration_s":1.477863,"scheduling_latency_s":1.477883,"redis_calls":13,"redis_duration_s":0.002966,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001032,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019519,"completed_at":"2026-07-25T11:30:06.186Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 268.248817] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.188Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6479ae2b6a476cf9ed6a4ce5","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6479ae2b6a476cf9ed6a4ce5: start","job_status":"start","queue_duration_s":1.498998,"scheduling_latency_s":1.499018} vm-test-run-nixbot-gitlab> gitlab # [ 268.261292] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.206Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6479ae2b6a476cf9ed6a4ce5","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6479ae2b6a476cf9ed6a4ce5: done: 0.018052 sec","job_status":"done","queue_duration_s":1.498998,"scheduling_latency_s":1.499018,"redis_calls":13,"redis_duration_s":0.0033420000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002288,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018052,"completed_at":"2026-07-25T11:30:06.206Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 268.300762] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.207Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93b770d0c2dfc70528c95452","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93b770d0c2dfc70528c95452: start","job_status":"start","queue_duration_s":1.518413,"scheduling_latency_s":1.51843} vm-test-run-nixbot-gitlab> gitlab # [ 268.313146] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93b770d0c2dfc70528c95452","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93b770d0c2dfc70528c95452: done: 0.016197 sec","job_status":"done","queue_duration_s":1.518413,"scheduling_latency_s":1.51843,"redis_calls":13,"redis_duration_s":0.002634,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001751,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016197,"completed_at":"2026-07-25T11:30:06.223Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 268.352169] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b99feb2941576d75cebe8173","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b99feb2941576d75cebe8173: start","job_status":"start","queue_duration_s":1.536838,"scheduling_latency_s":1.536857} vm-test-run-nixbot-gitlab> gitlab # [ 268.365667] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.242Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b99feb2941576d75cebe8173","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b99feb2941576d75cebe8173: done: 0.016713 sec","job_status":"done","queue_duration_s":1.536838,"scheduling_latency_s":1.536857,"redis_calls":13,"redis_duration_s":0.002764,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000944,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00182,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016713,"completed_at":"2026-07-25T11:30:06.242Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 268.405064] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.244Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df6662f563e87fdc528a844e","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df6662f563e87fdc528a844e: start","job_status":"start","queue_duration_s":1.554943,"scheduling_latency_s":1.554959} vm-test-run-nixbot-gitlab> gitlab # [ 268.417860] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df6662f563e87fdc528a844e","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df6662f563e87fdc528a844e: done: 0.016834 sec","job_status":"done","queue_duration_s":1.554943,"scheduling_latency_s":1.554959,"redis_calls":13,"redis_duration_s":0.002847,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000981,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001866,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010974,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016834,"completed_at":"2026-07-25T11:30:06.261Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 268.458076] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.262Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ecd5f526dd9186f2fc08bd88","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ecd5f526dd9186f2fc08bd88: start","job_status":"start","queue_duration_s":1.57326,"scheduling_latency_s":1.573278} vm-test-run-nixbot-gitlab> gitlab # [ 268.470662] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ecd5f526dd9186f2fc08bd88","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ecd5f526dd9186f2fc08bd88: done: 0.208117 sec","job_status":"done","queue_duration_s":1.57326,"scheduling_latency_s":1.573278,"redis_calls":13,"redis_duration_s":0.003241,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001744,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.208117,"completed_at":"2026-07-25T11:30:06.470Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 268.509842] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.475Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67f942ce7833de4a2f4acd33","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67f942ce7833de4a2f4acd33: start","job_status":"start","queue_duration_s":1.785696,"scheduling_latency_s":1.785754} vm-test-run-nixbot-gitlab> gitlab # [ 268.522444] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.519Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67f942ce7833de4a2f4acd33","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67f942ce7833de4a2f4acd33: done: 0.044744 sec","job_status":"done","queue_duration_s":1.785696,"scheduling_latency_s":1.785754,"redis_calls":13,"redis_duration_s":0.004527,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001695,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002832,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.044744,"completed_at":"2026-07-25T11:30:06.519Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 268.562021] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.522Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"999616e79635e0d118917094","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-999616e79635e0d118917094: start","job_status":"start","queue_duration_s":1.832976,"scheduling_latency_s":1.833013} vm-test-run-nixbot-gitlab> gitlab # [ 268.575700] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.554Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"999616e79635e0d118917094","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-999616e79635e0d118917094: done: 0.031978 sec","job_status":"done","queue_duration_s":1.832976,"scheduling_latency_s":1.833013,"redis_calls":13,"redis_duration_s":0.003992,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001268,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002724,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021208,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031978,"completed_at":"2026-07-25T11:30:06.554Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 268.615388] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ceb35d3bd204daf2f015b039","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ceb35d3bd204daf2f015b039: start","job_status":"start","queue_duration_s":1.867,"scheduling_latency_s":1.867029} vm-test-run-nixbot-gitlab> gitlab # [ 268.627841] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.584Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ceb35d3bd204daf2f015b039","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ceb35d3bd204daf2f015b039: done: 0.028547 sec","job_status":"done","queue_duration_s":1.867,"scheduling_latency_s":1.867029,"redis_calls":13,"redis_duration_s":0.004269,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001224,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003045,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018937,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028547,"completed_at":"2026-07-25T11:30:06.584Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 268.667624] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.587Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a45bf0bd749405863c1d7255","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a45bf0bd749405863c1d7255: start","job_status":"start","queue_duration_s":1.898278,"scheduling_latency_s":1.898304} vm-test-run-nixbot-gitlab> gitlab # [ 268.680254] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.607Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a45bf0bd749405863c1d7255","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a45bf0bd749405863c1d7255: done: 0.019571 sec","job_status":"done","queue_duration_s":1.898278,"scheduling_latency_s":1.898304,"redis_calls":13,"redis_duration_s":0.002878,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001013,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001865,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01247,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019571,"completed_at":"2026-07-25T11:30:06.607Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 268.719820] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.608Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e539a4ea03f667b6cf3df24","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e539a4ea03f667b6cf3df24: start","job_status":"start","queue_duration_s":1.919311,"scheduling_latency_s":1.91933} vm-test-run-nixbot-gitlab> gitlab # [ 268.732520] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.626Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e539a4ea03f667b6cf3df24","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e539a4ea03f667b6cf3df24: done: 0.017598 sec","job_status":"done","queue_duration_s":1.919311,"scheduling_latency_s":1.91933,"redis_calls":13,"redis_duration_s":0.002822,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000941,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001881,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011335,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017598,"completed_at":"2026-07-25T11:30:06.626Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 268.773102] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.627Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d2753ebcf3c3633e14a22ce","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d2753ebcf3c3633e14a22ce: start","job_status":"start","queue_duration_s":1.938346,"scheduling_latency_s":1.938364} vm-test-run-nixbot-gitlab> gitlab # [ 268.785719] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.644Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d2753ebcf3c3633e14a22ce","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d2753ebcf3c3633e14a22ce: done: 0.016519 sec","job_status":"done","queue_duration_s":1.938346,"scheduling_latency_s":1.938364,"redis_calls":13,"redis_duration_s":0.002715,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016519,"completed_at":"2026-07-25T11:30:06.644Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 268.825192] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.646Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e56316e951c767cae8f1c53f","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e56316e951c767cae8f1c53f: start","job_status":"start","queue_duration_s":1.956804,"scheduling_latency_s":1.956823} vm-test-run-nixbot-gitlab> gitlab # [ 268.837678] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e56316e951c767cae8f1c53f","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e56316e951c767cae8f1c53f: done: 0.016721 sec","job_status":"done","queue_duration_s":1.956804,"scheduling_latency_s":1.956823,"redis_calls":13,"redis_duration_s":0.003317,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001464,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001853,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016721,"completed_at":"2026-07-25T11:30:06.662Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 268.877134] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.664Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"981c76e5bd98a695c28358a5","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-981c76e5bd98a695c28358a5: start","job_status":"start","queue_duration_s":1.975019,"scheduling_latency_s":1.975036} vm-test-run-nixbot-gitlab> gitlab # [ 268.889478] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.681Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"981c76e5bd98a695c28358a5","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-981c76e5bd98a695c28358a5: done: 0.016907 sec","job_status":"done","queue_duration_s":1.975019,"scheduling_latency_s":1.975036,"redis_calls":13,"redis_duration_s":0.002742,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000924,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001818,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016907,"completed_at":"2026-07-25T11:30:06.681Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 268.928999] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.682Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a459a18fca2f719c28eaca54","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a459a18fca2f719c28eaca54: start","job_status":"start","queue_duration_s":1.993219,"scheduling_latency_s":1.993237} vm-test-run-nixbot-gitlab> gitlab # [ 268.941483] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.698Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a459a18fca2f719c28eaca54","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a459a18fca2f719c28eaca54: done: 0.016162 sec","job_status":"done","queue_duration_s":1.993219,"scheduling_latency_s":1.993237,"redis_calls":13,"redis_duration_s":0.002622,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000997,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001625,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010448,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016162,"completed_at":"2026-07-25T11:30:06.698Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 268.982094] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"840d5a4c59bb1f9df9a9e0ec","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-840d5a4c59bb1f9df9a9e0ec: start","job_status":"start","queue_duration_s":2.011247,"scheduling_latency_s":2.011267} vm-test-run-nixbot-gitlab> gitlab # [ 268.994647] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.717Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"840d5a4c59bb1f9df9a9e0ec","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-840d5a4c59bb1f9df9a9e0ec: done: 0.017114 sec","job_status":"done","queue_duration_s":2.011247,"scheduling_latency_s":2.011267,"redis_calls":13,"redis_duration_s":0.003262,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00143,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001832,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017114,"completed_at":"2026-07-25T11:30:06.717Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 269.033914] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.719Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3707e4842af09b1d4dd455d8","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3707e4842af09b1d4dd455d8: start","job_status":"start","queue_duration_s":2.029768,"scheduling_latency_s":2.029785} vm-test-run-nixbot-gitlab> gitlab # [ 269.046530] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.735Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3707e4842af09b1d4dd455d8","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3707e4842af09b1d4dd455d8: done: 0.01681 sec","job_status":"done","queue_duration_s":2.029768,"scheduling_latency_s":2.029785,"redis_calls":13,"redis_duration_s":0.0034400000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001041,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002399,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010869,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01681,"completed_at":"2026-07-25T11:30:06.735Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 269.086191] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc9ea931b4038c38caf2f154","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc9ea931b4038c38caf2f154: start","job_status":"start","queue_duration_s":2.048016,"scheduling_latency_s":2.048033} vm-test-run-nixbot-gitlab> gitlab # [ 269.098728] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.753Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc9ea931b4038c38caf2f154","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc9ea931b4038c38caf2f154: done: 0.015858 sec","job_status":"done","queue_duration_s":2.048016,"scheduling_latency_s":2.048033,"redis_calls":13,"redis_duration_s":0.003078,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001306,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001772,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015858,"completed_at":"2026-07-25T11:30:06.753Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 269.137970] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.754Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df3e79d5f01f744c87c7af80","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df3e79d5f01f744c87c7af80: start","job_status":"start","queue_duration_s":2.065212,"scheduling_latency_s":2.06523} vm-test-run-nixbot-gitlab> gitlab # [ 269.150588] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.770Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df3e79d5f01f744c87c7af80","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df3e79d5f01f744c87c7af80: done: 0.015707 sec","job_status":"done","queue_duration_s":2.065212,"scheduling_latency_s":2.06523,"redis_calls":13,"redis_duration_s":0.002567,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001662,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015707,"completed_at":"2026-07-25T11:30:06.770Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 269.191352] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.772Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30a0c9ebb5df19d775730ebd","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30a0c9ebb5df19d775730ebd: start","job_status":"start","queue_duration_s":2.082734,"scheduling_latency_s":2.082752} vm-test-run-nixbot-gitlab> gitlab # [ 269.204068] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30a0c9ebb5df19d775730ebd","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30a0c9ebb5df19d775730ebd: done: 0.016654 sec","job_status":"done","queue_duration_s":2.082734,"scheduling_latency_s":2.082752,"redis_calls":13,"redis_duration_s":0.002718,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001783,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016654,"completed_at":"2026-07-25T11:30:06.788Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 269.243520] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66308c63945764c169b531fb","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66308c63945764c169b531fb: start","job_status":"start","queue_duration_s":2.100827,"scheduling_latency_s":2.100845} vm-test-run-nixbot-gitlab> gitlab # [ 269.255854] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66308c63945764c169b531fb","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66308c63945764c169b531fb: done: 0.016939 sec","job_status":"done","queue_duration_s":2.100827,"scheduling_latency_s":2.100845,"redis_calls":13,"redis_duration_s":0.0027749999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000949,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001826,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016939,"completed_at":"2026-07-25T11:30:06.807Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 269.296007] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.808Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"873dd3c5f020ece289efd0c9","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-873dd3c5f020ece289efd0c9: start","job_status":"start","queue_duration_s":2.11939,"scheduling_latency_s":2.119408} vm-test-run-nixbot-gitlab> gitlab # [ 269.308458] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.825Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"873dd3c5f020ece289efd0c9","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-873dd3c5f020ece289efd0c9: done: 0.016756 sec","job_status":"done","queue_duration_s":2.11939,"scheduling_latency_s":2.119408,"redis_calls":13,"redis_duration_s":0.002788,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000982,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001806,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010737,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016756,"completed_at":"2026-07-25T11:30:06.825Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 269.347984] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9c7adfc4c8bb0c84797f08d","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9c7adfc4c8bb0c84797f08d: start","job_status":"start","queue_duration_s":2.137553,"scheduling_latency_s":2.137572} vm-test-run-nixbot-gitlab> gitlab # [ 269.360656] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9c7adfc4c8bb0c84797f08d","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9c7adfc4c8bb0c84797f08d: done: 0.01689 sec","job_status":"done","queue_duration_s":2.137553,"scheduling_latency_s":2.137572,"redis_calls":13,"redis_duration_s":0.002862,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001004,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001858,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010425,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01689,"completed_at":"2026-07-25T11:30:06.843Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 269.400771] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.845Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d1e39fa3fe03f8e9a17b2607","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d1e39fa3fe03f8e9a17b2607: start","job_status":"start","queue_duration_s":2.156149,"scheduling_latency_s":2.156167} vm-test-run-nixbot-gitlab> gitlab # [ 269.413368] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d1e39fa3fe03f8e9a17b2607","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d1e39fa3fe03f8e9a17b2607: done: 0.016535 sec","job_status":"done","queue_duration_s":2.156149,"scheduling_latency_s":2.156167,"redis_calls":13,"redis_duration_s":0.002696,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001763,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010655,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016535,"completed_at":"2026-07-25T11:30:06.862Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 269.452951] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.863Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"442237d0452f13d7ac47dc88","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-442237d0452f13d7ac47dc88: start","job_status":"start","queue_duration_s":2.174055,"scheduling_latency_s":2.174072} vm-test-run-nixbot-gitlab> gitlab # [ 269.465725] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.879Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"442237d0452f13d7ac47dc88","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-442237d0452f13d7ac47dc88: done: 0.016056 sec","job_status":"done","queue_duration_s":2.174055,"scheduling_latency_s":2.174072,"redis_calls":13,"redis_duration_s":0.002806,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000962,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001844,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016056,"completed_at":"2026-07-25T11:30:06.879Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 269.505165] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.880Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"447ece04f109dbea39e4b9ec","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-447ece04f109dbea39e4b9ec: start","job_status":"start","queue_duration_s":2.191456,"scheduling_latency_s":2.191473} vm-test-run-nixbot-gitlab> gitlab # [ 269.517592] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"447ece04f109dbea39e4b9ec","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-447ece04f109dbea39e4b9ec: done: 0.015724 sec","job_status":"done","queue_duration_s":2.191456,"scheduling_latency_s":2.191473,"redis_calls":13,"redis_duration_s":0.00262,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001721,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015724,"completed_at":"2026-07-25T11:30:06.896Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 269.557172] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d1021281d7ceaddcbc93d982","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d1021281d7ceaddcbc93d982: start","job_status":"start","queue_duration_s":2.209119,"scheduling_latency_s":2.209136} vm-test-run-nixbot-gitlab> gitlab # [ 269.569670] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d1021281d7ceaddcbc93d982","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d1021281d7ceaddcbc93d982: done: 0.016802 sec","job_status":"done","queue_duration_s":2.209119,"scheduling_latency_s":2.209136,"redis_calls":13,"redis_duration_s":0.003167,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002213,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016802,"completed_at":"2026-07-25T11:30:06.915Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 269.610300] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0903ee43d6109f6c4565bfac","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0903ee43d6109f6c4565bfac: start","job_status":"start","queue_duration_s":2.227391,"scheduling_latency_s":2.22741} vm-test-run-nixbot-gitlab> gitlab # [ 269.622766] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0903ee43d6109f6c4565bfac","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0903ee43d6109f6c4565bfac: done: 0.016553 sec","job_status":"done","queue_duration_s":2.227391,"scheduling_latency_s":2.22741,"redis_calls":13,"redis_duration_s":0.002834,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001827,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016553,"completed_at":"2026-07-25T11:30:06.933Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 269.662048] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c17529da863c4bd39e13e4c0","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c17529da863c4bd39e13e4c0: start","job_status":"start","queue_duration_s":2.245385,"scheduling_latency_s":2.245405} vm-test-run-nixbot-gitlab> gitlab # [ 269.674698] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:30:06.952Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c17529da863c4bd39e13e4c0","created_at":"2026-07-25T11:30:04.583Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-ba6dae8529b8403f","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"318e41de5556cb402048edadf5147cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:30:04.689Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c17529da863c4bd39e13e4c0: done: 0.018205 sec","job_status":"done","queue_duration_s":2.245385,"scheduling_latency_s":2.245405,"redis_calls":13,"redis_duration_s":0.002797,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000988,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001809,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018205,"completed_at":"2026-07-25T11:30:06.952Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 301.608594] setup-gitlab-start[3691]: W, [2026-07-25T11:30:41.420513 #3691] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 310.274491] redis[1324]: 10 changes in 300 seconds. Saving... vm-test-run-nixbot-gitlab> gitlab # [ 310.280000] redis[1324]: Background saving started by pid 3916 vm-test-run-nixbot-gitlab> gitlab # [ 310.286029] redis[3916]: BGSAVE done, 527 keys saved, 0 keys skipped, 130856 bytes written. vm-test-run-nixbot-gitlab> gitlab # [ 310.290617] redis[3916]: DB saved on disk vm-test-run-nixbot-gitlab> gitlab # [ 310.292941] redis[3916]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB vm-test-run-nixbot-gitlab> gitlab # [ 310.375657] redis[1324]: Background saving terminated with success vm-test-run-nixbot-gitlab> gitlab # [ 311.648318] postgres[1408]: [1408] LOG: checkpoint starting: time vm-test-run-nixbot-gitlab> gitlab # [ 313.285008] gitaly[1447]: time="2026-07-25T11:30:53.096Z" level=info msg="burden monitor stats" pid=1447 top_10_by_cpu="[]" top_10_by_memory="[]" vm-test-run-nixbot-gitlab> gitlab # [ 322.604901] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:02.415Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b5ba8f0abc3745ed8a15d227","created_at":"2026-07-25T11:31:02.396Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"24ef6fcbfafd8f782167ba9b80b3294c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-25T11:31:02.408Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b5ba8f0abc3745ed8a15d227: start","job_status":"start","queue_duration_s":0.006175,"scheduling_latency_s":0.006231} vm-test-run-nixbot-gitlab> gitlab # [ 323.290006] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:03.101Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b5ba8f0abc3745ed8a15d227","created_at":"2026-07-25T11:31:02.396Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"24ef6fcbfafd8f782167ba9b80b3294c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-25T11:31:02.408Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b5ba8f0abc3745ed8a15d227: done: 0.686991 sec","job_status":"done","queue_duration_s":0.006175,"scheduling_latency_s":0.006231,"redis_calls":15,"redis_duration_s":0.087423,"redis_read_bytes":1029,"redis_write_bytes":1306,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.047681,"redis_feature_flag_read_bytes":1024,"redis_feature_flag_write_bytes":565,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.02384,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.015902,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074749,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.686991,"completed_at":"2026-07-25T11:31:03.101Z","load_balancing_strategy":"primary","db_duration_s":0.25006,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 323.403448] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:03.216Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"1eaa0ec35572f89ad1771fcd","created_at":"2026-07-25T11:31:02.459Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a65b25a133850202083df3a97dd11aa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:02.544Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-1eaa0ec35572f89ad1771fcd: start","job_status":"start","queue_duration_s":0.671727,"scheduling_latency_s":0.671762} vm-test-run-nixbot-gitlab> gitlab # [ 323.867738] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:03.678Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"1eaa0ec35572f89ad1771fcd","created_at":"2026-07-25T11:31:02.459Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a65b25a133850202083df3a97dd11aa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:02.544Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-1eaa0ec35572f89ad1771fcd: done: 0.462322 sec","job_status":"done","queue_duration_s":0.671727,"scheduling_latency_s":0.671762,"redis_calls":16,"redis_duration_s":0.121038,"redis_read_bytes":272,"redis_write_bytes":4229,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.008933,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":199,"redis_queues_calls":2,"redis_queues_duration_s":0.002379,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.098585,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.011141,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.066162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.462322,"completed_at":"2026-07-25T11:31:03.678Z","load_balancing_strategy":"primary","db_duration_s":0.01965,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 323.958324] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:03.758Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"dbc206d9c6528d5981a83cc3","created_at":"2026-07-25T11:31:02.573Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5f92f4c1a63e4b1d604dc3accd39fc3b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:02.588Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-dbc206d9c6528d5981a83cc3: start","job_status":"start","queue_duration_s":1.169769,"scheduling_latency_s":1.169835} vm-test-run-nixbot-gitlab> gitlab # [ 324.288316] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.100Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"dbc206d9c6528d5981a83cc3","created_at":"2026-07-25T11:31:02.573Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5f92f4c1a63e4b1d604dc3accd39fc3b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:02.588Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-dbc206d9c6528d5981a83cc3: done: 0.341909 sec","job_status":"done","queue_duration_s":1.169769,"scheduling_latency_s":1.169835,"redis_calls":11,"redis_duration_s":0.01573,"redis_read_bytes":263,"redis_write_bytes":1334,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000816,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":172,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001876,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.013038,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.034,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001181,"exclusive_lock_hold_duration_s":0.083272,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.341909,"completed_at":"2026-07-25T11:31:04.099Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.063687,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 324.380351] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.104Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"d840cdfe6909b0a012946951","created_at":"2026-07-25T11:31:02.624Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0d7139bb56c940789ac29c9cdc404d0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:02.653Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-d840cdfe6909b0a012946951: start","job_status":"start","queue_duration_s":1.450665,"scheduling_latency_s":1.450714} vm-test-run-nixbot-gitlab> gitlab # [ 324.397153] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.137Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"d840cdfe6909b0a012946951","created_at":"2026-07-25T11:31:02.624Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0d7139bb56c940789ac29c9cdc404d0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:02.653Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-d840cdfe6909b0a012946951: done: 0.033711 sec","job_status":"done","queue_duration_s":1.450665,"scheduling_latency_s":1.450714,"redis_calls":9,"redis_duration_s":0.003476,"redis_read_bytes":261,"redis_write_bytes":944,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000654,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":168,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002089,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000733,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022357,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033711,"completed_at":"2026-07-25T11:31:04.137Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 324.440267] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.140Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"a68f5d6fdbb97fafb9233628","created_at":"2026-07-25T11:31:02.706Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ebb42d0d52805c8cad770ee5bdd4750","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-25T11:31:02.721Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-a68f5d6fdbb97fafb9233628: start","job_status":"start","queue_duration_s":1.418424,"scheduling_latency_s":1.418456} vm-test-run-nixbot-gitlab> gitlab # [ 324.453881] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.153Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 324.457650] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.160Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"a68f5d6fdbb97fafb9233628","created_at":"2026-07-25T11:31:02.706Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ebb42d0d52805c8cad770ee5bdd4750","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-25T11:31:02.721Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-a68f5d6fdbb97fafb9233628: done: 0.020963 sec","job_status":"done","queue_duration_s":1.418424,"scheduling_latency_s":1.418456,"redis_calls":8,"redis_duration_s":0.002215,"redis_read_bytes":133,"redis_write_bytes":908,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000249,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":107,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001415,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000551,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020963,"completed_at":"2026-07-25T11:31:04.160Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 324.500654] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.162Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"5d78f478ed2602aeab16db1c","created_at":"2026-07-25T11:31:02.749Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4493e4c6f992fde7d76b489501b2e440","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:02.770Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-5d78f478ed2602aeab16db1c: start","job_status":"start","queue_duration_s":1.392108,"scheduling_latency_s":1.39213} vm-test-run-nixbot-gitlab> gitlab # [ 324.515385] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.206Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"5d78f478ed2602aeab16db1c","created_at":"2026-07-25T11:31:02.749Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4493e4c6f992fde7d76b489501b2e440","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:02.770Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-5d78f478ed2602aeab16db1c: done: 0.044055 sec","job_status":"done","queue_duration_s":1.392108,"scheduling_latency_s":1.39213,"redis_calls":10,"redis_duration_s":0.003009,"redis_read_bytes":135,"redis_write_bytes":1184,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000238,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":97,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001399,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001372,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01457,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00071014,"exclusive_lock_hold_duration_s":0.00503066,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.044055,"completed_at":"2026-07-25T11:31:04.206Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002878,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 324.560221] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.208Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"ca2e11efb917925e7e3e7264","created_at":"2026-07-25T11:31:02.838Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"15a8d42076cec72af136784efb36bc0d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:02.859Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-ca2e11efb917925e7e3e7264: start","job_status":"start","queue_duration_s":1.349287,"scheduling_latency_s":1.349312} vm-test-run-nixbot-gitlab> gitlab # [ 324.574379] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.246Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"ca2e11efb917925e7e3e7264","created_at":"2026-07-25T11:31:02.838Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"15a8d42076cec72af136784efb36bc0d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:02.859Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-ca2e11efb917925e7e3e7264: done: 0.037253 sec","job_status":"done","queue_duration_s":1.349287,"scheduling_latency_s":1.349312,"redis_calls":8,"redis_duration_s":0.002607,"redis_read_bytes":133,"redis_write_bytes":936,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000242,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":110,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001724,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000641,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.037253,"completed_at":"2026-07-25T11:31:04.246Z","load_balancing_strategy":"primary","db_duration_s":0.015507,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 324.618094] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.248Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"a4761cf53953260cadf87c22","created_at":"2026-07-25T11:31:02.898Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9bce1211b595f43be9f641682b5f6a36","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:02.964Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-a4761cf53953260cadf87c22: start","job_status":"start","queue_duration_s":1.284259,"scheduling_latency_s":1.284288} vm-test-run-nixbot-gitlab> gitlab # [ 324.713976] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.526Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"a4761cf53953260cadf87c22","created_at":"2026-07-25T11:31:02.898Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9bce1211b595f43be9f641682b5f6a36","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:02.964Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-a4761cf53953260cadf87c22: done: 0.278176 sec","job_status":"done","queue_duration_s":1.284259,"scheduling_latency_s":1.284288,"redis_calls":10,"redis_duration_s":0.0032849999999999997,"redis_read_bytes":135,"redis_write_bytes":1071,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.00027,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":84,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001563,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001452,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":123,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":123,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.051,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.051,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.210112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00080358,"exclusive_lock_hold_duration_s":0.25662926,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.278176,"completed_at":"2026-07-25T11:31:04.526Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.077448,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 324.790964] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.528Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"fd3949c49e3963399737d290","created_at":"2026-07-25T11:31:03.018Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1cfb8912c40f3d4378d41eecb1ba47ab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.044Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-fd3949c49e3963399737d290: start","job_status":"start","queue_duration_s":1.484254,"scheduling_latency_s":1.484276} vm-test-run-nixbot-gitlab> gitlab # [ 324.807543] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.586Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"fd3949c49e3963399737d290","created_at":"2026-07-25T11:31:03.018Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1cfb8912c40f3d4378d41eecb1ba47ab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.044Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-fd3949c49e3963399737d290: done: 0.057691 sec","job_status":"done","queue_duration_s":1.484254,"scheduling_latency_s":1.484276,"redis_calls":10,"redis_duration_s":0.003183,"redis_read_bytes":135,"redis_write_bytes":1275,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000215,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":106,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001433,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001535,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0009015,"exclusive_lock_hold_duration_s":0.02469598,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.057691,"completed_at":"2026-07-25T11:31:04.585Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031113,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 324.854520] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.588Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"f58db61a99e6ee205904921b","created_at":"2026-07-25T11:31:03.075Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3718a7c3396df32569268fa987eff176","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.092Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-f58db61a99e6ee205904921b: start","job_status":"start","queue_duration_s":1.495845,"scheduling_latency_s":1.49587} vm-test-run-nixbot-gitlab> gitlab # [ 324.869932] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.598Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 324.873253] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.604Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"f58db61a99e6ee205904921b","created_at":"2026-07-25T11:31:03.075Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3718a7c3396df32569268fa987eff176","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.092Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-f58db61a99e6ee205904921b: done: 0.016526 sec","job_status":"done","queue_duration_s":1.495845,"scheduling_latency_s":1.49587,"redis_calls":8,"redis_duration_s":0.002259,"redis_read_bytes":133,"redis_write_bytes":873,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000235,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":102,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00138,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000644,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016526,"completed_at":"2026-07-25T11:31:04.604Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 324.917674] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.606Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"7382109ea8e96f2412309bc0","created_at":"2026-07-25T11:31:03.236Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f15294b0edfd6e60df830a6d87ffcaf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.248Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-7382109ea8e96f2412309bc0: start","job_status":"start","queue_duration_s":1.357323,"scheduling_latency_s":1.357341} vm-test-run-nixbot-gitlab> gitlab # [ 324.932144] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.615Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 324.935657] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.621Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"7382109ea8e96f2412309bc0","created_at":"2026-07-25T11:31:03.236Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f15294b0edfd6e60df830a6d87ffcaf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.248Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-7382109ea8e96f2412309bc0: done: 0.015042 sec","job_status":"done","queue_duration_s":1.357323,"scheduling_latency_s":1.357341,"redis_calls":8,"redis_duration_s":0.001977,"redis_read_bytes":133,"redis_write_bytes":867,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000243,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":101,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001245,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009225,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015042,"completed_at":"2026-07-25T11:31:04.621Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 324.978493] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.624Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"d541a26298e9fc50934740c6","created_at":"2026-07-25T11:31:03.276Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"493d780c94a3252155779f830e25c75d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.293Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-d541a26298e9fc50934740c6: start","job_status":"start","queue_duration_s":1.331276,"scheduling_latency_s":1.331326} vm-test-run-nixbot-gitlab> gitlab # [ 324.993406] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.648Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 324.997205] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.657Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"d541a26298e9fc50934740c6","created_at":"2026-07-25T11:31:03.276Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"493d780c94a3252155779f830e25c75d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.293Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-d541a26298e9fc50934740c6: done: 0.032763 sec","job_status":"done","queue_duration_s":1.331276,"scheduling_latency_s":1.331326,"redis_calls":8,"redis_duration_s":0.0031009999999999996,"redis_read_bytes":133,"redis_write_bytes":931,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000402,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":110,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001969,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00073,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032763,"completed_at":"2026-07-25T11:31:04.657Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 325.039914] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.659Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"c37dadf41e9ea3b071b8e8ca","created_at":"2026-07-25T11:31:03.320Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13551161a49bdc7f496b5b8bca8186d1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.334Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-c37dadf41e9ea3b071b8e8ca: start","job_status":"start","queue_duration_s":1.324494,"scheduling_latency_s":1.324519} vm-test-run-nixbot-gitlab> gitlab # [ 325.054682] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.676Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"c37dadf41e9ea3b071b8e8ca","created_at":"2026-07-25T11:31:03.320Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13551161a49bdc7f496b5b8bca8186d1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.334Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-c37dadf41e9ea3b071b8e8ca: done: 0.017599 sec","job_status":"done","queue_duration_s":1.324494,"scheduling_latency_s":1.324519,"redis_calls":8,"redis_duration_s":0.001995,"redis_read_bytes":133,"redis_write_bytes":943,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000233,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":112,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001273,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017599,"completed_at":"2026-07-25T11:31:04.676Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 325.097602] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.678Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"70af38b568bb880884b0d663","created_at":"2026-07-25T11:31:03.378Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ccd86b3a423e7cdec90e41f42a6c26a3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.395Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-70af38b568bb880884b0d663: start","job_status":"start","queue_duration_s":1.283509,"scheduling_latency_s":1.283534} vm-test-run-nixbot-gitlab> gitlab # [ 325.113421] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.688Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 325.116807] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.694Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"70af38b568bb880884b0d663","created_at":"2026-07-25T11:31:03.378Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ccd86b3a423e7cdec90e41f42a6c26a3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.395Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-70af38b568bb880884b0d663: done: 0.015742 sec","job_status":"done","queue_duration_s":1.283509,"scheduling_latency_s":1.283534,"redis_calls":8,"redis_duration_s":0.0020080000000000002,"redis_read_bytes":133,"redis_write_bytes":937,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000227,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":111,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001257,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000524,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009653,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015742,"completed_at":"2026-07-25T11:31:04.694Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 325.159782] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.697Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"a319b0c2d59679d1cb0f6648","created_at":"2026-07-25T11:31:03.438Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4383fd68305dda812c494614093aa180","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.468Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-a319b0c2d59679d1cb0f6648: start","job_status":"start","queue_duration_s":1.228802,"scheduling_latency_s":1.228855} vm-test-run-nixbot-gitlab> gitlab # [ 325.174629] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.740Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"a319b0c2d59679d1cb0f6648","created_at":"2026-07-25T11:31:03.438Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4383fd68305dda812c494614093aa180","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.468Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-a319b0c2d59679d1cb0f6648: done: 0.043223 sec","job_status":"done","queue_duration_s":1.228802,"scheduling_latency_s":1.228855,"redis_calls":11,"redis_duration_s":0.0041519999999999994,"redis_read_bytes":136,"redis_write_bytes":1339,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000271,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":100,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002225,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001656,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0079914,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.043223,"completed_at":"2026-07-25T11:31:04.740Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004321,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 325.219442] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.742Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"96765c1940a147cefe595fb8","created_at":"2026-07-25T11:31:03.542Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fddcb9a6cd8734406fe5bf9360626594","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-25T11:31:03.563Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-96765c1940a147cefe595fb8: start","job_status":"start","queue_duration_s":1.178721,"scheduling_latency_s":1.178754} vm-test-run-nixbot-gitlab> gitlab # [ 325.233187] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.756Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 325.236494] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.764Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"96765c1940a147cefe595fb8","created_at":"2026-07-25T11:31:03.542Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fddcb9a6cd8734406fe5bf9360626594","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-25T11:31:03.563Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-96765c1940a147cefe595fb8: done: 0.021772 sec","job_status":"done","queue_duration_s":1.178721,"scheduling_latency_s":1.178754,"redis_calls":8,"redis_duration_s":0.002249,"redis_read_bytes":133,"redis_write_bytes":914,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000269,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":108,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001415,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000565,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021772,"completed_at":"2026-07-25T11:31:04.764Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 325.278147] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.766Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"19eab938425b5c722dfa8a90","created_at":"2026-07-25T11:31:03.477Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-1a249fa008674fd8","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3a65b25a133850202083df3a97dd11aa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.587Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-19eab938425b5c722dfa8a90: start","job_status":"start","queue_duration_s":1.179675,"scheduling_latency_s":1.179709} vm-test-run-nixbot-gitlab> gitlab # [ 325.290604] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.788Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 325.292298] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.801Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"19eab938425b5c722dfa8a90","created_at":"2026-07-25T11:31:03.477Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-1a249fa008674fd8","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3a65b25a133850202083df3a97dd11aa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.587Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-19eab938425b5c722dfa8a90: done: 0.034812 sec","job_status":"done","queue_duration_s":1.179675,"scheduling_latency_s":1.179709,"redis_calls":11,"redis_duration_s":0.004223,"redis_read_bytes":136,"redis_write_bytes":1104,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000267,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":82,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001397,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002559,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034812,"completed_at":"2026-07-25T11:31:04.801Z","load_balancing_strategy":"primary","db_duration_s":0.005177,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 325.333288] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.803Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4b174c254a76200909a54425","created_at":"2026-07-25T11:31:03.477Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-1a249fa008674fd8","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3a65b25a133850202083df3a97dd11aa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.587Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4b174c254a76200909a54425: start","job_status":"start","queue_duration_s":1.2165,"scheduling_latency_s":1.216532} vm-test-run-nixbot-gitlab> gitlab # [ 325.345665] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.817Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 325.347346] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.827Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4b174c254a76200909a54425","created_at":"2026-07-25T11:31:03.477Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-1a249fa008674fd8","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3a65b25a133850202083df3a97dd11aa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.587Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4b174c254a76200909a54425: done: 0.024219 sec","job_status":"done","queue_duration_s":1.2165,"scheduling_latency_s":1.216532,"redis_calls":10,"redis_duration_s":0.002836,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001183,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001653,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014603,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024219,"completed_at":"2026-07-25T11:31:04.827Z","load_balancing_strategy":"primary","db_duration_s":0.003331,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 325.386316] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.829Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"30242187fc5f76b3c82c22b6","created_at":"2026-07-25T11:31:03.477Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-1a249fa008674fd8","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3a65b25a133850202083df3a97dd11aa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.587Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-30242187fc5f76b3c82c22b6: start","job_status":"start","queue_duration_s":1.242468,"scheduling_latency_s":1.242497} vm-test-run-nixbot-gitlab> gitlab # [ 325.398843] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.842Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 325.400566] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.852Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"30242187fc5f76b3c82c22b6","created_at":"2026-07-25T11:31:03.477Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-1a249fa008674fd8","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3a65b25a133850202083df3a97dd11aa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.587Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-30242187fc5f76b3c82c22b6: done: 0.022718 sec","job_status":"done","queue_duration_s":1.242468,"scheduling_latency_s":1.242497,"redis_calls":10,"redis_duration_s":0.0028139999999999997,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001166,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001648,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022718,"completed_at":"2026-07-25T11:31:04.852Z","load_balancing_strategy":"primary","db_duration_s":0.00292,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 325.439575] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.853Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ac3e12b27c3a521201301dfb","created_at":"2026-07-25T11:31:03.477Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-1a249fa008674fd8","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3a65b25a133850202083df3a97dd11aa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.587Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ac3e12b27c3a521201301dfb: start","job_status":"start","queue_duration_s":1.266902,"scheduling_latency_s":1.266933} vm-test-run-nixbot-gitlab> gitlab # [ 325.452380] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.866Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 325.454148] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.875Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ac3e12b27c3a521201301dfb","created_at":"2026-07-25T11:31:03.477Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-1a249fa008674fd8","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3a65b25a133850202083df3a97dd11aa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.587Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ac3e12b27c3a521201301dfb: done: 0.021831 sec","job_status":"done","queue_duration_s":1.266902,"scheduling_latency_s":1.266933,"redis_calls":10,"redis_duration_s":0.002676,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001117,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001559,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021831,"completed_at":"2026-07-25T11:31:04.875Z","load_balancing_strategy":"primary","db_duration_s":0.002815,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 325.492634] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:04.877Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"ee6b8ef96faae9c01294af33","created_at":"2026-07-25T11:31:03.623Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4c81c09a005607fbfba9e023c755caec","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.639Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-ee6b8ef96faae9c01294af33: start","job_status":"start","queue_duration_s":1.237767,"scheduling_latency_s":1.237797} vm-test-run-nixbot-gitlab> gitlab # [ 325.506780] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.203Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"ee6b8ef96faae9c01294af33","created_at":"2026-07-25T11:31:03.623Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4c81c09a005607fbfba9e023c755caec","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.639Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-ee6b8ef96faae9c01294af33: done: 0.325868 sec","job_status":"done","queue_duration_s":1.237767,"scheduling_latency_s":1.237797,"redis_calls":572,"redis_duration_s":0.067108,"redis_read_bytes":697,"redis_write_bytes":49570,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000229,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":83,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001462,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.065417,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.294967,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.325868,"completed_at":"2026-07-25T11:31:05.203Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 325.550100] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.204Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"330ece89636d18e2c2b9a54a","created_at":"2026-07-25T11:31:03.667Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.744Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-330ece89636d18e2c2b9a54a: start","job_status":"start","queue_duration_s":1.460379,"scheduling_latency_s":1.460406} vm-test-run-nixbot-gitlab> gitlab # [ 325.563698] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.304Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"330ece89636d18e2c2b9a54a","created_at":"2026-07-25T11:31:03.667Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.744Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-330ece89636d18e2c2b9a54a: done: 0.099506 sec","job_status":"done","queue_duration_s":1.460379,"scheduling_latency_s":1.460406,"redis_calls":65,"redis_duration_s":0.010199,"redis_read_bytes":748,"redis_write_bytes":41402,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.000574,"redis_feature_flag_read_bytes":640,"redis_feature_flag_write_bytes":393,"redis_queues_calls":2,"redis_queues_duration_s":0.000728,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37385,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.008151,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000746,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.090343,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.099506,"completed_at":"2026-07-25T11:31:05.304Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 325.606610] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.305Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"736475220d0f5712c3fd34e0","created_at":"2026-07-25T11:31:03.782Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ea09dd72726d2efbaa23582040d8af3","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.797Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-736475220d0f5712c3fd34e0: start","job_status":"start","queue_duration_s":1.507903,"scheduling_latency_s":1.507928} vm-test-run-nixbot-gitlab> gitlab # [ 325.621050] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.335Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"736475220d0f5712c3fd34e0","created_at":"2026-07-25T11:31:03.782Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ea09dd72726d2efbaa23582040d8af3","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.797Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-736475220d0f5712c3fd34e0: done: 0.029382 sec","job_status":"done","queue_duration_s":1.507903,"scheduling_latency_s":1.507928,"redis_calls":11,"redis_duration_s":0.001718,"redis_read_bytes":517,"redis_write_bytes":1011,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000602,"redis_feature_flag_read_bytes":512,"redis_feature_flag_write_bytes":300,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000803,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000313,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029382,"completed_at":"2026-07-25T11:31:05.335Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008309,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 325.663259] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.336Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"58aef9eae1838378faa31fbc","created_at":"2026-07-25T11:31:03.854Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50612898344c00ba3a77ddd969a8978f","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.870Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-58aef9eae1838378faa31fbc: start","job_status":"start","queue_duration_s":1.465867,"scheduling_latency_s":1.46589} vm-test-run-nixbot-gitlab> gitlab # [ 325.677516] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.354Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"58aef9eae1838378faa31fbc","created_at":"2026-07-25T11:31:03.854Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50612898344c00ba3a77ddd969a8978f","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.870Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-58aef9eae1838378faa31fbc: done: 0.017948 sec","job_status":"done","queue_duration_s":1.465867,"scheduling_latency_s":1.46589,"redis_calls":8,"redis_duration_s":0.0013109999999999999,"redis_read_bytes":133,"redis_write_bytes":764,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000144,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":83,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000842,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000325,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017948,"completed_at":"2026-07-25T11:31:05.354Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001717,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 325.720251] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.355Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"b500e5b28ae9c24a5d15db80","created_at":"2026-07-25T11:31:03.934Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfcf5a12881faafce8fdeef90715c774","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.958Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-b500e5b28ae9c24a5d15db80: start","job_status":"start","queue_duration_s":1.397248,"scheduling_latency_s":1.397271} vm-test-run-nixbot-gitlab> gitlab # [ 325.733850] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.375Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"b500e5b28ae9c24a5d15db80","created_at":"2026-07-25T11:31:03.934Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-563f609c43bc4e7c","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfcf5a12881faafce8fdeef90715c774","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:03.958Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-b500e5b28ae9c24a5d15db80: done: 0.019808 sec","job_status":"done","queue_duration_s":1.397248,"scheduling_latency_s":1.397271,"redis_calls":10,"redis_duration_s":0.0018310000000000002,"redis_read_bytes":135,"redis_write_bytes":1167,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.00015,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":95,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000814,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012851,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00059802,"exclusive_lock_hold_duration_s":0.003878,"duration_s":0.019808,"completed_at":"2026-07-25T11:31:05.375Z","load_balancing_strategy":"primary","db_duration_s":0.001771,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 325.775533] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af274b06adfcd98171e40439","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af274b06adfcd98171e40439: start","job_status":"start","queue_duration_s":0.081126,"scheduling_latency_s":0.081144} vm-test-run-nixbot-gitlab> gitlab # [ 325.788167] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.393Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af274b06adfcd98171e40439","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af274b06adfcd98171e40439: done: 0.016955 sec","job_status":"done","queue_duration_s":0.081126,"scheduling_latency_s":0.081144,"redis_calls":14,"redis_duration_s":0.0018609999999999998,"redis_read_bytes":138,"redis_write_bytes":1242,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000135,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":85,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000588,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001138,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016955,"completed_at":"2026-07-25T11:31:05.393Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 325.829066] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b611888d7a55f20d5f4082a3","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b611888d7a55f20d5f4082a3: start","job_status":"start","queue_duration_s":0.099326,"scheduling_latency_s":0.099345} vm-test-run-nixbot-gitlab> gitlab # [ 325.841305] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.411Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b611888d7a55f20d5f4082a3","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b611888d7a55f20d5f4082a3: done: 0.016281 sec","job_status":"done","queue_duration_s":0.099326,"scheduling_latency_s":0.099345,"redis_calls":13,"redis_duration_s":0.001955,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001053,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016281,"completed_at":"2026-07-25T11:31:05.411Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 325.883578] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.412Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b04070ceada679fc029fcdf9","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b04070ceada679fc029fcdf9: start","job_status":"start","queue_duration_s":0.116762,"scheduling_latency_s":0.11678} vm-test-run-nixbot-gitlab> gitlab # [ 325.896596] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b04070ceada679fc029fcdf9","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b04070ceada679fc029fcdf9: done: 0.016401 sec","job_status":"done","queue_duration_s":0.116762,"scheduling_latency_s":0.11678,"redis_calls":13,"redis_duration_s":0.0018089999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000635,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016401,"completed_at":"2026-07-25T11:31:05.428Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 325.938204] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fffe7ef52445fe0ce0fa2360","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fffe7ef52445fe0ce0fa2360: start","job_status":"start","queue_duration_s":0.134562,"scheduling_latency_s":0.134585} vm-test-run-nixbot-gitlab> gitlab # [ 325.951006] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.448Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fffe7ef52445fe0ce0fa2360","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fffe7ef52445fe0ce0fa2360: done: 0.017783 sec","job_status":"done","queue_duration_s":0.134562,"scheduling_latency_s":0.134585,"redis_calls":13,"redis_duration_s":0.001859,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000698,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017783,"completed_at":"2026-07-25T11:31:05.447Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 325.991335] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4850d8f01b31d61e02182caa","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4850d8f01b31d61e02182caa: start","job_status":"start","queue_duration_s":0.153716,"scheduling_latency_s":0.153736} vm-test-run-nixbot-gitlab> gitlab # [ 326.004209] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4850d8f01b31d61e02182caa","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4850d8f01b31d61e02182caa: done: 0.018023 sec","job_status":"done","queue_duration_s":0.153716,"scheduling_latency_s":0.153736,"redis_calls":13,"redis_duration_s":0.0018909999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000697,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018023,"completed_at":"2026-07-25T11:31:05.467Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 326.044167] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e14ba396000fc9bb4156991f","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e14ba396000fc9bb4156991f: start","job_status":"start","queue_duration_s":0.172864,"scheduling_latency_s":0.172884} vm-test-run-nixbot-gitlab> gitlab # [ 326.056891] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.483Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e14ba396000fc9bb4156991f","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e14ba396000fc9bb4156991f: done: 0.01491 sec","job_status":"done","queue_duration_s":0.172864,"scheduling_latency_s":0.172884,"redis_calls":13,"redis_duration_s":0.001465,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01491,"completed_at":"2026-07-25T11:31:05.483Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 326.098379] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab5b3e4c27bc05aad01aa417","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab5b3e4c27bc05aad01aa417: start","job_status":"start","queue_duration_s":0.18885,"scheduling_latency_s":0.188867} vm-test-run-nixbot-gitlab> gitlab # [ 326.111360] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.499Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab5b3e4c27bc05aad01aa417","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab5b3e4c27bc05aad01aa417: done: 0.014882 sec","job_status":"done","queue_duration_s":0.18885,"scheduling_latency_s":0.188867,"redis_calls":13,"redis_duration_s":0.001521,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000967,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01025,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014882,"completed_at":"2026-07-25T11:31:05.499Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 326.152889] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.500Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc5a4897ea0828db8394aef7","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc5a4897ea0828db8394aef7: start","job_status":"start","queue_duration_s":0.204836,"scheduling_latency_s":0.204854} vm-test-run-nixbot-gitlab> gitlab # [ 326.165590] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.515Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc5a4897ea0828db8394aef7","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc5a4897ea0828db8394aef7: done: 0.015316 sec","job_status":"done","queue_duration_s":0.204836,"scheduling_latency_s":0.204854,"redis_calls":13,"redis_duration_s":0.001505,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010255,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015316,"completed_at":"2026-07-25T11:31:05.515Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 326.205369] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.516Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a16b55782157d50b66387a18","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a16b55782157d50b66387a18: start","job_status":"start","queue_duration_s":0.221236,"scheduling_latency_s":0.221253} vm-test-run-nixbot-gitlab> gitlab # [ 326.218133] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.531Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a16b55782157d50b66387a18","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a16b55782157d50b66387a18: done: 0.014922 sec","job_status":"done","queue_duration_s":0.221236,"scheduling_latency_s":0.221253,"redis_calls":13,"redis_duration_s":0.0015470000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000981,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014922,"completed_at":"2026-07-25T11:31:05.531Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 326.258741] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.533Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c76f5645ddfa02464cab8c43","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c76f5645ddfa02464cab8c43: start","job_status":"start","queue_duration_s":0.237282,"scheduling_latency_s":0.2373} vm-test-run-nixbot-gitlab> gitlab # [ 326.271762] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.547Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c76f5645ddfa02464cab8c43","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c76f5645ddfa02464cab8c43: done: 0.014743 sec","job_status":"done","queue_duration_s":0.237282,"scheduling_latency_s":0.2373,"redis_calls":13,"redis_duration_s":0.001493,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000961,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014743,"completed_at":"2026-07-25T11:31:05.547Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 326.312507] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.548Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd5df1d9a4f88d06af9e1e79","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd5df1d9a4f88d06af9e1e79: start","job_status":"start","queue_duration_s":0.253128,"scheduling_latency_s":0.253145} vm-test-run-nixbot-gitlab> gitlab # [ 326.325427] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.563Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd5df1d9a4f88d06af9e1e79","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd5df1d9a4f88d06af9e1e79: done: 0.014991 sec","job_status":"done","queue_duration_s":0.253128,"scheduling_latency_s":0.253145,"redis_calls":13,"redis_duration_s":0.001578,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000999,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010284,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014991,"completed_at":"2026-07-25T11:31:05.563Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 326.366720] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.565Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e8901c375f653610872fe9f","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e8901c375f653610872fe9f: start","job_status":"start","queue_duration_s":0.269324,"scheduling_latency_s":0.269342} vm-test-run-nixbot-gitlab> gitlab # [ 326.379628] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.580Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e8901c375f653610872fe9f","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e8901c375f653610872fe9f: done: 0.015624 sec","job_status":"done","queue_duration_s":0.269324,"scheduling_latency_s":0.269342,"redis_calls":13,"redis_duration_s":0.001742,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000624,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010707,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015624,"completed_at":"2026-07-25T11:31:05.580Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 326.420261] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.581Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08cbe9d5344af1bf6798ac19","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08cbe9d5344af1bf6798ac19: start","job_status":"start","queue_duration_s":0.286222,"scheduling_latency_s":0.286242} vm-test-run-nixbot-gitlab> gitlab # [ 326.432832] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.598Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08cbe9d5344af1bf6798ac19","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08cbe9d5344af1bf6798ac19: done: 0.016085 sec","job_status":"done","queue_duration_s":0.286222,"scheduling_latency_s":0.286242,"redis_calls":13,"redis_duration_s":0.0017270000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00063,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016085,"completed_at":"2026-07-25T11:31:05.597Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 326.472523] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.599Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d194b55d6e662043eb013768","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d194b55d6e662043eb013768: start","job_status":"start","queue_duration_s":0.303484,"scheduling_latency_s":0.303502} vm-test-run-nixbot-gitlab> gitlab # [ 326.485072] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.614Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d194b55d6e662043eb013768","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d194b55d6e662043eb013768: done: 0.015232 sec","job_status":"done","queue_duration_s":0.303484,"scheduling_latency_s":0.303502,"redis_calls":13,"redis_duration_s":0.001509,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010315,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015232,"completed_at":"2026-07-25T11:31:05.614Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 326.524605] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc211c218f10928f17374288","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc211c218f10928f17374288: start","job_status":"start","queue_duration_s":0.319809,"scheduling_latency_s":0.319827} vm-test-run-nixbot-gitlab> gitlab # [ 326.538054] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc211c218f10928f17374288","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc211c218f10928f17374288: done: 0.014649 sec","job_status":"done","queue_duration_s":0.319809,"scheduling_latency_s":0.319827,"redis_calls":13,"redis_duration_s":0.0014889999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014649,"completed_at":"2026-07-25T11:31:05.630Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 326.577536] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.631Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e23570fb2e6b3ffb527d399","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e23570fb2e6b3ffb527d399: start","job_status":"start","queue_duration_s":0.335592,"scheduling_latency_s":0.335611} vm-test-run-nixbot-gitlab> gitlab # [ 326.590127] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.646Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e23570fb2e6b3ffb527d399","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e23570fb2e6b3ffb527d399: done: 0.015131 sec","job_status":"done","queue_duration_s":0.335592,"scheduling_latency_s":0.335611,"redis_calls":13,"redis_duration_s":0.0015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015131,"completed_at":"2026-07-25T11:31:05.646Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 326.629483] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"466c1023307e7f6ed469a4d9","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-466c1023307e7f6ed469a4d9: start","job_status":"start","queue_duration_s":0.351842,"scheduling_latency_s":0.35186} vm-test-run-nixbot-gitlab> gitlab # [ 326.641975] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"466c1023307e7f6ed469a4d9","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-466c1023307e7f6ed469a4d9: done: 0.014947 sec","job_status":"done","queue_duration_s":0.351842,"scheduling_latency_s":0.35186,"redis_calls":13,"redis_duration_s":0.001547,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014947,"completed_at":"2026-07-25T11:31:05.662Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 326.681282] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.663Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e46b9cad2ae751016611309","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e46b9cad2ae751016611309: start","job_status":"start","queue_duration_s":0.36792,"scheduling_latency_s":0.367938} vm-test-run-nixbot-gitlab> gitlab # [ 326.693530] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e46b9cad2ae751016611309","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e46b9cad2ae751016611309: done: 0.014591 sec","job_status":"done","queue_duration_s":0.36792,"scheduling_latency_s":0.367938,"redis_calls":13,"redis_duration_s":0.00149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014591,"completed_at":"2026-07-25T11:31:05.678Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 326.732917] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40a0b4d9750ba8382cd6ae6e","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40a0b4d9750ba8382cd6ae6e: start","job_status":"start","queue_duration_s":0.383651,"scheduling_latency_s":0.38367} vm-test-run-nixbot-gitlab> gitlab # [ 326.746663] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.694Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40a0b4d9750ba8382cd6ae6e","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40a0b4d9750ba8382cd6ae6e: done: 0.014742 sec","job_status":"done","queue_duration_s":0.383651,"scheduling_latency_s":0.38367,"redis_calls":13,"redis_duration_s":0.001506,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010178,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014742,"completed_at":"2026-07-25T11:31:05.694Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 326.786207] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd88381044cf74ffe32cdb88","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd88381044cf74ffe32cdb88: start","job_status":"start","queue_duration_s":0.399516,"scheduling_latency_s":0.399535} vm-test-run-nixbot-gitlab> gitlab # [ 326.798803] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.710Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd88381044cf74ffe32cdb88","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd88381044cf74ffe32cdb88: done: 0.015231 sec","job_status":"done","queue_duration_s":0.399516,"scheduling_latency_s":0.399535,"redis_calls":13,"redis_duration_s":0.001744,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001211,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010398,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015231,"completed_at":"2026-07-25T11:31:05.710Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 326.838556] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35fc6bb57eba5792a174aa7c","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35fc6bb57eba5792a174aa7c: start","job_status":"start","queue_duration_s":0.415894,"scheduling_latency_s":0.415912} vm-test-run-nixbot-gitlab> gitlab # [ 326.851034] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35fc6bb57eba5792a174aa7c","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35fc6bb57eba5792a174aa7c: done: 0.015625 sec","job_status":"done","queue_duration_s":0.415894,"scheduling_latency_s":0.415912,"redis_calls":13,"redis_duration_s":0.001573,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010814,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015625,"completed_at":"2026-07-25T11:31:05.727Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 326.891579] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.728Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2efd2a09ec8dbc3b033638f","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2efd2a09ec8dbc3b033638f: start","job_status":"start","queue_duration_s":0.432663,"scheduling_latency_s":0.432681} vm-test-run-nixbot-gitlab> gitlab # [ 326.904433] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2efd2a09ec8dbc3b033638f","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2efd2a09ec8dbc3b033638f: done: 0.015122 sec","job_status":"done","queue_duration_s":0.432663,"scheduling_latency_s":0.432681,"redis_calls":13,"redis_duration_s":0.001568,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015122,"completed_at":"2026-07-25T11:31:05.743Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 326.944992] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.744Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aceea19b9964bc83bc8abb55","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aceea19b9964bc83bc8abb55: start","job_status":"start","queue_duration_s":0.448961,"scheduling_latency_s":0.44898} vm-test-run-nixbot-gitlab> gitlab # [ 326.957603] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aceea19b9964bc83bc8abb55","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aceea19b9964bc83bc8abb55: done: 0.015109 sec","job_status":"done","queue_duration_s":0.448961,"scheduling_latency_s":0.44898,"redis_calls":13,"redis_duration_s":0.001613,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010294,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015109,"completed_at":"2026-07-25T11:31:05.759Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 326.996830] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d02cab3041ba3736f9ef4a4","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d02cab3041ba3736f9ef4a4: start","job_status":"start","queue_duration_s":0.465161,"scheduling_latency_s":0.465177} vm-test-run-nixbot-gitlab> gitlab # [ 327.009288] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d02cab3041ba3736f9ef4a4","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d02cab3041ba3736f9ef4a4: done: 0.014578 sec","job_status":"done","queue_duration_s":0.465161,"scheduling_latency_s":0.465177,"redis_calls":13,"redis_duration_s":0.001473,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014578,"completed_at":"2026-07-25T11:31:05.775Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 327.048527] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.776Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19c061e7ea6d4c97a2057b52","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19c061e7ea6d4c97a2057b52: start","job_status":"start","queue_duration_s":0.480821,"scheduling_latency_s":0.48084} vm-test-run-nixbot-gitlab> gitlab # [ 327.060954] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19c061e7ea6d4c97a2057b52","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19c061e7ea6d4c97a2057b52: done: 0.014962 sec","job_status":"done","queue_duration_s":0.480821,"scheduling_latency_s":0.48084,"redis_calls":13,"redis_duration_s":0.001581,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000592,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010238,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014962,"completed_at":"2026-07-25T11:31:05.791Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 327.100342] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.792Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c239716a569d878ea0b1ae8b","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c239716a569d878ea0b1ae8b: start","job_status":"start","queue_duration_s":0.496922,"scheduling_latency_s":0.49694} vm-test-run-nixbot-gitlab> gitlab # [ 327.112723] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c239716a569d878ea0b1ae8b","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c239716a569d878ea0b1ae8b: done: 0.015242 sec","job_status":"done","queue_duration_s":0.496922,"scheduling_latency_s":0.49694,"redis_calls":13,"redis_duration_s":0.0018110000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000811,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015242,"completed_at":"2026-07-25T11:31:05.807Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 327.153732] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1deaa4a680add44a9227d811","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1deaa4a680add44a9227d811: start","job_status":"start","queue_duration_s":0.513303,"scheduling_latency_s":0.513322} vm-test-run-nixbot-gitlab> gitlab # [ 327.166440] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1deaa4a680add44a9227d811","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1deaa4a680add44a9227d811: done: 0.015578 sec","job_status":"done","queue_duration_s":0.513303,"scheduling_latency_s":0.513322,"redis_calls":13,"redis_duration_s":0.0017029999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000632,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015578,"completed_at":"2026-07-25T11:31:05.824Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 327.206107] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.825Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f611d687b705f437d8d779b","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f611d687b705f437d8d779b: start","job_status":"start","queue_duration_s":0.530209,"scheduling_latency_s":0.530231} vm-test-run-nixbot-gitlab> gitlab # [ 327.218652] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.841Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f611d687b705f437d8d779b","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f611d687b705f437d8d779b: done: 0.016078 sec","job_status":"done","queue_duration_s":0.530209,"scheduling_latency_s":0.530231,"redis_calls":13,"redis_duration_s":0.001751,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00066,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016078,"completed_at":"2026-07-25T11:31:05.841Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 327.258126] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6d03580872677df7966a844","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6d03580872677df7966a844: start","job_status":"start","queue_duration_s":0.547743,"scheduling_latency_s":0.547765} vm-test-run-nixbot-gitlab> gitlab # [ 327.270787] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.860Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6d03580872677df7966a844","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6d03580872677df7966a844: done: 0.016789 sec","job_status":"done","queue_duration_s":0.547743,"scheduling_latency_s":0.547765,"redis_calls":13,"redis_duration_s":0.0017200000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000646,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001074,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016789,"completed_at":"2026-07-25T11:31:05.860Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 327.310304] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c38ad2249049a1b4e4a2f62","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c38ad2249049a1b4e4a2f62: start","job_status":"start","queue_duration_s":0.56662,"scheduling_latency_s":0.566644} vm-test-run-nixbot-gitlab> gitlab # [ 327.323092] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c38ad2249049a1b4e4a2f62","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c38ad2249049a1b4e4a2f62: done: 0.021874 sec","job_status":"done","queue_duration_s":0.56662,"scheduling_latency_s":0.566644,"redis_calls":13,"redis_duration_s":0.00199,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021874,"completed_at":"2026-07-25T11:31:05.884Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 327.364295] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.886Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca329ac16dfa0d36d71f84ae","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca329ac16dfa0d36d71f84ae: start","job_status":"start","queue_duration_s":0.590753,"scheduling_latency_s":0.590788} vm-test-run-nixbot-gitlab> gitlab # [ 327.377007] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca329ac16dfa0d36d71f84ae","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca329ac16dfa0d36d71f84ae: done: 0.02973 sec","job_status":"done","queue_duration_s":0.590753,"scheduling_latency_s":0.590788,"redis_calls":13,"redis_duration_s":0.003444,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001037,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002407,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02973,"completed_at":"2026-07-25T11:31:05.916Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 327.416767] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe708ba3816944ecb9d35736","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe708ba3816944ecb9d35736: start","job_status":"start","queue_duration_s":0.62241,"scheduling_latency_s":0.622435} vm-test-run-nixbot-gitlab> gitlab # [ 327.429299] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.944Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe708ba3816944ecb9d35736","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe708ba3816944ecb9d35736: done: 0.026283 sec","job_status":"done","queue_duration_s":0.62241,"scheduling_latency_s":0.622435,"redis_calls":13,"redis_duration_s":0.002389,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000764,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001625,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018547,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026283,"completed_at":"2026-07-25T11:31:05.944Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 327.468602] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef3909f021d5011978a61956","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef3909f021d5011978a61956: start","job_status":"start","queue_duration_s":0.649985,"scheduling_latency_s":0.650006} vm-test-run-nixbot-gitlab> gitlab # [ 327.481173] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef3909f021d5011978a61956","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef3909f021d5011978a61956: done: 0.021274 sec","job_status":"done","queue_duration_s":0.649985,"scheduling_latency_s":0.650006,"redis_calls":13,"redis_duration_s":0.0023569999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000611,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001746,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015203,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021274,"completed_at":"2026-07-25T11:31:05.966Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 327.520779] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.968Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"782648287a3c0d9f35765359","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-782648287a3c0d9f35765359: start","job_status":"start","queue_duration_s":0.673211,"scheduling_latency_s":0.673233} vm-test-run-nixbot-gitlab> gitlab # [ 327.533297] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"782648287a3c0d9f35765359","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-782648287a3c0d9f35765359: done: 0.026961 sec","job_status":"done","queue_duration_s":0.673211,"scheduling_latency_s":0.673233,"redis_calls":13,"redis_duration_s":0.002791,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019064,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026961,"completed_at":"2026-07-25T11:31:05.995Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 327.574157] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:05.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"348b869ec47476d891284cbf","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-348b869ec47476d891284cbf: start","job_status":"start","queue_duration_s":0.701527,"scheduling_latency_s":0.701549} vm-test-run-nixbot-gitlab> gitlab # [ 327.586937] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"348b869ec47476d891284cbf","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-348b869ec47476d891284cbf: done: 0.023277 sec","job_status":"done","queue_duration_s":0.701527,"scheduling_latency_s":0.701549,"redis_calls":13,"redis_duration_s":0.002428,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000853,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001575,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023277,"completed_at":"2026-07-25T11:31:06.020Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 327.628253] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.022Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"083da24e666732137d16fa61","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-083da24e666732137d16fa61: start","job_status":"start","queue_duration_s":0.727119,"scheduling_latency_s":0.727143} vm-test-run-nixbot-gitlab> gitlab # [ 327.640889] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.049Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"083da24e666732137d16fa61","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-083da24e666732137d16fa61: done: 0.026754 sec","job_status":"done","queue_duration_s":0.727119,"scheduling_latency_s":0.727143,"redis_calls":13,"redis_duration_s":0.002923,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001914,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026754,"completed_at":"2026-07-25T11:31:06.049Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 327.680784] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a74c803f25dd675ef16446ec","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a74c803f25dd675ef16446ec: start","job_status":"start","queue_duration_s":0.756063,"scheduling_latency_s":0.756101} vm-test-run-nixbot-gitlab> gitlab # [ 327.693362] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a74c803f25dd675ef16446ec","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a74c803f25dd675ef16446ec: done: 0.062845 sec","job_status":"done","queue_duration_s":0.756063,"scheduling_latency_s":0.756101,"redis_calls":13,"redis_duration_s":0.0058980000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001199,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004699,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.062845,"completed_at":"2026-07-25T11:31:06.114Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 327.732927] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5bb1d8aebf66bc5e5193d52","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5bb1d8aebf66bc5e5193d52: start","job_status":"start","queue_duration_s":0.821958,"scheduling_latency_s":0.822046} vm-test-run-nixbot-gitlab> gitlab # [ 327.745482] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.169Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5bb1d8aebf66bc5e5193d52","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5bb1d8aebf66bc5e5193d52: done: 0.051611 sec","job_status":"done","queue_duration_s":0.821958,"scheduling_latency_s":0.822046,"redis_calls":13,"redis_duration_s":0.004572,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.051611,"completed_at":"2026-07-25T11:31:06.169Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 327.788146] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c898f11a935fc04f86945d1d","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c898f11a935fc04f86945d1d: start","job_status":"start","queue_duration_s":0.876949,"scheduling_latency_s":0.877011} vm-test-run-nixbot-gitlab> gitlab # [ 327.800636] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c898f11a935fc04f86945d1d","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c898f11a935fc04f86945d1d: done: 0.054048 sec","job_status":"done","queue_duration_s":0.876949,"scheduling_latency_s":0.877011,"redis_calls":13,"redis_duration_s":0.0046,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001162,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003438,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.054048,"completed_at":"2026-07-25T11:31:06.226Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 327.840301] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"860565d0eda841ddb8b6d803","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-860565d0eda841ddb8b6d803: start","job_status":"start","queue_duration_s":0.933941,"scheduling_latency_s":0.933984} vm-test-run-nixbot-gitlab> gitlab # [ 327.852770] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.270Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"860565d0eda841ddb8b6d803","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-860565d0eda841ddb8b6d803: done: 0.041253 sec","job_status":"done","queue_duration_s":0.933941,"scheduling_latency_s":0.933984,"redis_calls":13,"redis_duration_s":0.003974,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001394,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00258,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031384,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.041253,"completed_at":"2026-07-25T11:31:06.270Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 327.892417] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d5cfdc012c1d17f44c7a593","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d5cfdc012c1d17f44c7a593: start","job_status":"start","queue_duration_s":0.97745,"scheduling_latency_s":0.977475} vm-test-run-nixbot-gitlab> gitlab # [ 327.904925] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.314Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d5cfdc012c1d17f44c7a593","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d5cfdc012c1d17f44c7a593: done: 0.0417 sec","job_status":"done","queue_duration_s":0.97745,"scheduling_latency_s":0.977475,"redis_calls":13,"redis_duration_s":0.00442,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001719,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002701,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0417,"completed_at":"2026-07-25T11:31:06.314Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 327.944749] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d327e9ad6429814fe623ae41","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d327e9ad6429814fe623ae41: start","job_status":"start","queue_duration_s":1.022342,"scheduling_latency_s":1.02237} vm-test-run-nixbot-gitlab> gitlab # [ 327.957332] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.359Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d327e9ad6429814fe623ae41","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d327e9ad6429814fe623ae41: done: 0.041471 sec","job_status":"done","queue_duration_s":1.022342,"scheduling_latency_s":1.02237,"redis_calls":13,"redis_duration_s":0.003882,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001283,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002599,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.041471,"completed_at":"2026-07-25T11:31:06.359Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 327.998313] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.361Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0094bfd2fbabcef190aeae21","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0094bfd2fbabcef190aeae21: start","job_status":"start","queue_duration_s":1.06583,"scheduling_latency_s":1.065865} vm-test-run-nixbot-gitlab> gitlab # [ 328.011002] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0094bfd2fbabcef190aeae21","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0094bfd2fbabcef190aeae21: done: 0.038387 sec","job_status":"done","queue_duration_s":1.06583,"scheduling_latency_s":1.065865,"redis_calls":13,"redis_duration_s":0.003322,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001134,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002188,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038387,"completed_at":"2026-07-25T11:31:06.399Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 328.050918] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.402Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44c0f40e2790714cd1aaedbd","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44c0f40e2790714cd1aaedbd: start","job_status":"start","queue_duration_s":1.106486,"scheduling_latency_s":1.106509} vm-test-run-nixbot-gitlab> gitlab # [ 328.063628] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.440Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44c0f40e2790714cd1aaedbd","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44c0f40e2790714cd1aaedbd: done: 0.038609 sec","job_status":"done","queue_duration_s":1.106486,"scheduling_latency_s":1.106509,"redis_calls":13,"redis_duration_s":0.003366,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001113,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002253,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029045,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038609,"completed_at":"2026-07-25T11:31:06.440Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 328.103475] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.443Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"229d126ce90c972d0b0a5c2b","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-229d126ce90c972d0b0a5c2b: start","job_status":"start","queue_duration_s":1.14815,"scheduling_latency_s":1.14818} vm-test-run-nixbot-gitlab> gitlab # [ 328.116257] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"229d126ce90c972d0b0a5c2b","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-229d126ce90c972d0b0a5c2b: done: 0.042539 sec","job_status":"done","queue_duration_s":1.14815,"scheduling_latency_s":1.14818,"redis_calls":13,"redis_duration_s":0.0034539999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001151,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002303,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030762,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.042539,"completed_at":"2026-07-25T11:31:06.486Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 328.156217] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.489Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60fc8779c58ad2afbdfa44d3","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60fc8779c58ad2afbdfa44d3: start","job_status":"start","queue_duration_s":1.194089,"scheduling_latency_s":1.194165} vm-test-run-nixbot-gitlab> gitlab # [ 328.170100] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.597Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60fc8779c58ad2afbdfa44d3","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60fc8779c58ad2afbdfa44d3: done: 0.10797 sec","job_status":"done","queue_duration_s":1.194089,"scheduling_latency_s":1.194165,"redis_calls":13,"redis_duration_s":0.004087,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001577,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00251,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.094908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.10797,"completed_at":"2026-07-25T11:31:06.597Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 328.209739] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57721b39213d763066dfd507","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57721b39213d763066dfd507: start","job_status":"start","queue_duration_s":1.304852,"scheduling_latency_s":1.304886} vm-test-run-nixbot-gitlab> gitlab # [ 328.222174] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.643Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57721b39213d763066dfd507","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57721b39213d763066dfd507: done: 0.042469 sec","job_status":"done","queue_duration_s":1.304852,"scheduling_latency_s":1.304886,"redis_calls":13,"redis_duration_s":0.003933,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001338,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002595,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.042469,"completed_at":"2026-07-25T11:31:06.642Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 328.261510] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d4d020c22e1464830b542b2","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d4d020c22e1464830b542b2: start","job_status":"start","queue_duration_s":1.352376,"scheduling_latency_s":1.352421} vm-test-run-nixbot-gitlab> gitlab # [ 328.274065] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.694Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d4d020c22e1464830b542b2","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d4d020c22e1464830b542b2: done: 0.046768 sec","job_status":"done","queue_duration_s":1.352376,"scheduling_latency_s":1.352421,"redis_calls":13,"redis_duration_s":0.003972,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001415,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002557,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.046768,"completed_at":"2026-07-25T11:31:06.694Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 328.313628] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9c45f3589bb3480ec555498","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9c45f3589bb3480ec555498: start","job_status":"start","queue_duration_s":1.401865,"scheduling_latency_s":1.401921} vm-test-run-nixbot-gitlab> gitlab # [ 328.326204] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9c45f3589bb3480ec555498","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9c45f3589bb3480ec555498: done: 0.040563 sec","job_status":"done","queue_duration_s":1.401865,"scheduling_latency_s":1.401921,"redis_calls":13,"redis_duration_s":0.0029249999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001068,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001857,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02891,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.040563,"completed_at":"2026-07-25T11:31:06.737Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 328.365624] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.740Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"380d71b83cc1a4b0349a3011","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-380d71b83cc1a4b0349a3011: start","job_status":"start","queue_duration_s":1.44482,"scheduling_latency_s":1.444856} vm-test-run-nixbot-gitlab> gitlab # [ 328.379351] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:06.786Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"380d71b83cc1a4b0349a3011","created_at":"2026-07-25T11:31:05.217Z","trace_propagation_headers":{"sentry-trace":"6cc8d16dfd2d4e43a1838a094c1d6f9f-4ddbacac931849d6","baggage":"sentry-trace_id=6cc8d16dfd2d4e43a1838a094c1d6f9f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"03a48a9b8a2423fbea37698345fddb02","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:05.295Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-380d71b83cc1a4b0349a3011: done: 0.045517 sec","job_status":"done","queue_duration_s":1.44482,"scheduling_latency_s":1.444856,"redis_calls":13,"redis_duration_s":0.003894,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001329,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002565,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.045517,"completed_at":"2026-07-25T11:31:06.785Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 344.230892] runuser[3690]: pam_unix(runuser:session): session closed for user gitlab vm-test-run-nixbot-gitlab> gitlab # [ 344.234882] setup-gitlab-start[4055]: +++ date -d '+7 days' +%F vm-test-run-nixbot-gitlab> gitlab # [ 344.235817] setup-gitlab-start[4056]: ++ jq -r .token vm-test-run-nixbot-gitlab> gitlab # [ 344.241338] setup-gitlab-start[4054]: ++ curl -fs -X POST -H 'Authorization: Bearer 716aee96aeecb4756eb506a74f49c247de01a387adbb9a1e7559a28c762259ac' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-08-01"}' http://gitlab/api/v4/users/1/personal_access_tokens vm-test-run-nixbot-gitlab> gitlab # [ 344.911905] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:24.722Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"92205267e669143ec3edd632","created_at":"2026-07-25T11:31:24.683Z","trace_propagation_headers":{"sentry-trace":"0c753b4a7ea24c5581161ad723d4b84a-22dcfe5048734b7f","baggage":"sentry-trace_id=0c753b4a7ea24c5581161ad723d4b84a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGPMSF881ZKWX8X01K2JNT","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:5efc37909c5126be2aaaa03c0679320bb507caa87e8544a5a0ac0f3c8d1c0dd6","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:24.718Z","job_size_bytes":451,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-92205267e669143ec3edd632: start","job_status":"start","queue_duration_s":0.004504,"scheduling_latency_s":0.004594} vm-test-run-nixbot-gitlab> gitlab # [ 344.961364] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:24 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 683 vm-test-run-nixbot-gitlab> gitlab # [ 344.962184] setup-gitlab-start[3635]: + TOKEN=glpat-APv49i6FJ9XKayZ3Zq8-4286MQp1OjEH.01.0w0qt1xta vm-test-run-nixbot-gitlab> gitlab # [ 344.962324] setup-gitlab-start[3635]: + test -n glpat-APv49i6FJ9XKayZ3Zq8-4286MQp1OjEH.01.0w0qt1xta vm-test-run-nixbot-gitlab> gitlab # [ 344.962431] setup-gitlab-start[3635]: + test glpat-APv49i6FJ9XKayZ3Zq8-4286MQp1OjEH.01.0w0qt1xta '!=' null vm-test-run-nixbot-gitlab> gitlab # [ 344.962525] setup-gitlab-start[3635]: + mkdir -p /var/lib/secrets vm-test-run-nixbot-gitlab> gitlab # [ 344.962626] setup-gitlab-start[3635]: + echo glpat-APv49i6FJ9XKayZ3Zq8-4286MQp1OjEH.01.0w0qt1xta vm-test-run-nixbot-gitlab> gitlab # [ 344.962717] setup-gitlab-start[3635]: + chmod 644 /var/lib/secrets/gitlab-token vm-test-run-nixbot-gitlab> gitlab # [ 344.966065] setup-gitlab-start[3635]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-APv49i6FJ9XKayZ3Zq8-4286MQp1OjEH.01.0w0qt1xta' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects vm-test-run-nixbot-gitlab> gitlab # [ 345.131279] sidekiq-cluster[2673]: {"severity":"WARN","time":"2026-07-25T11:31:24.942Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"92205267e669143ec3edd632","created_at":"2026-07-25T11:31:24.683Z","trace_propagation_headers":{"sentry-trace":"0c753b4a7ea24c5581161ad723d4b84a-22dcfe5048734b7f","baggage":"sentry-trace_id=0c753b4a7ea24c5581161ad723d4b84a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGPMSF881ZKWX8X01K2JNT","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:5efc37909c5126be2aaaa03c0679320bb507caa87e8544a5a0ac0f3c8d1c0dd6","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:24.718Z","job_size_bytes":451,"pid":2673,"sidekiq_tid":"lyeh","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-92205267e669143ec3edd632: fail: 0.219169 sec","job_status":"fail","queue_duration_s":0.004504,"scheduling_latency_s":0.004594,"redis_calls":11,"redis_duration_s":0.004893,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001413,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002545,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.152342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00066994,"duration_s":0.219169,"completed_at":"2026-07-25T11:31:24.941Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/w9mp1gyfs2qdv12r0ji70idzndmxwz2s-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/p8ylyr0sj9sxcknqcpja13nypqixk2dp-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/dndr7b4a17339dsm67vzb6nv4p2kp82q-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/zx6ndpscxs64hnidz69ab430ccms6rak-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.045586,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 345.824404] gitaly[1447]: time="2026-07-25T11:31:25.636Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPNG3K1D68V0J58NCWGWK error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-25T11:31:35.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:25.635" pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 345.855988] gitaly[1447]: time="2026-07-25T11:31:25.637Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPNG3K1D68V0J58NCWGWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-25T11:31:35.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:25.635" grpc.time_ms=1.071 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 346.036853] gitaly[1447]: time="2026-07-25T11:31:25.849Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPNG3K1D68V0J58NCWGWK error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-25T11:31:35.218" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:25.849" pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 346.068890] gitaly[1447]: time="2026-07-25T11:31:25.850Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPNG3K1D68V0J58NCWGWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-25T11:31:35.218" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:25.849" grpc.time_ms=0.567 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 346.103707] gitaly[1447]: time="2026-07-25T11:31:25.894Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPNG3K1D68V0J58NCWGWK diskcache=b077846e-4ffa-455d-9e79-31e69edd31b0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-25T11:31:35.230" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:25.861" pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 346.127871] gitaly[1447]: time="2026-07-25T11:31:25.895Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=299 command.oublock=32 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPNG3K1D68V0J58NCWGWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-25T11:31:35.230" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:25.861" grpc.time_ms=33.931 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 346.154689] gitaly[1447]: time="2026-07-25T11:31:25.913Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPNG3K1D68V0J58NCWGWK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-25T11:31:35.281" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:25.912" pid=1447 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 346.172993] gitaly[1447]: time="2026-07-25T11:31:25.914Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPNG3K1D68V0J58NCWGWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-25T11:31:35.281" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:25.912" grpc.time_ms=1.453 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 346.190318] gitaly[1447]: time="2026-07-25T11:31:25.940Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPNG3K1D68V0J58NCWGWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-25T11:31:35.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:25.922" grpc.time_ms=17.428 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 346.211027] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:25.986Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b10ac6131cd3f84bafc0a44f","created_at":"2026-07-25T11:31:25.957Z","trace_propagation_headers":{"sentry-trace":"304b49c0dffa41f1a9832a65ffb85956-65ff8f1dd8874741","baggage":"sentry-trace_id=304b49c0dffa41f1a9832a65ffb85956,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGPNG3K1D68V0J58NCWGWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:25.977Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b10ac6131cd3f84bafc0a44f: start","job_status":"start","queue_duration_s":0.009167,"scheduling_latency_s":0.009231} vm-test-run-nixbot-gitlab> gitlab # [ 346.229138] gitaly[1447]: time="2026-07-25T11:31:26.030Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPNG3K1D68V0J58NCWGWK error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-25T11:31:36.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:26.030" pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 346.240670] gitaly[1447]: time="2026-07-25T11:31:26.031Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPNG3K1D68V0J58NCWGWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-25T11:31:36.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:26.030" grpc.time_ms=0.634 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 346.256840] gitaly[1447]: time="2026-07-25T11:31:26.070Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPNG3K1D68V0J58NCWGWK diskcache=8c4de027-d4a0-40ca-bc9b-cd86af808836 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-25T11:31:36.015" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:26.036" pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 346.268193] gitaly[1447]: time="2026-07-25T11:31:26.070Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=299 command.oublock=32 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPNG3K1D68V0J58NCWGWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-25T11:31:36.015" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:26.036" grpc.time_ms=33.791 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 346.282619] gitaly[1447]: time="2026-07-25T11:31:26.084Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPNG3K1D68V0J58NCWGWK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-25T11:31:36.062" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:26.083" pid=1447 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 346.293524] gitaly[1447]: time="2026-07-25T11:31:26.085Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPNG3K1D68V0J58NCWGWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-25T11:31:36.062" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:26.083" grpc.time_ms=1.33 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 346.304840] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:26.086Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b10ac6131cd3f84bafc0a44f","created_at":"2026-07-25T11:31:25.957Z","trace_propagation_headers":{"sentry-trace":"304b49c0dffa41f1a9832a65ffb85956-65ff8f1dd8874741","baggage":"sentry-trace_id=304b49c0dffa41f1a9832a65ffb85956,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGPNG3K1D68V0J58NCWGWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:25.977Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b10ac6131cd3f84bafc0a44f: done: 0.099848 sec","job_status":"done","queue_duration_s":0.009167,"scheduling_latency_s":0.009231,"redis_calls":13,"redis_duration_s":0.004649,"redis_read_bytes":138,"redis_write_bytes":1593,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001159,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":428,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001899,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001591,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.073115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.05078174,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.099848,"completed_at":"2026-07-25T11:31:26.086Z","load_balancing_strategy":"primary","db_duration_s":0.013081,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 346.351457] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:26.088Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"1acdec921050a4ceea82d1bf","created_at":"2026-07-25T11:31:26.004Z","trace_propagation_headers":{"sentry-trace":"304b49c0dffa41f1a9832a65ffb85956-65ff8f1dd8874741","baggage":"sentry-trace_id=304b49c0dffa41f1a9832a65ffb85956,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGPNG3K1D68V0J58NCWGWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:26.019Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-1acdec921050a4ceea82d1bf: start","job_status":"start","queue_duration_s":0.068621,"scheduling_latency_s":0.068647} vm-test-run-nixbot-gitlab> gitlab # [ 346.366491] gitaly[1447]: time="2026-07-25T11:31:26.114Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPNG3K1D68V0J58NCWGWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-25T11:31:36.072" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:26.092" grpc.time_ms=21.625 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 346.380542] gitaly[1447]: time="2026-07-25T11:31:26.175Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPNG3K1D68V0J58NCWGWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-25T11:31:36.154" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:26.174" grpc.time_ms=0.259 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 346.391432] gitaly[1447]: time="2026-07-25T11:31:26.186Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPNG3K1D68V0J58NCWGWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-25T11:31:36.165" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:26.185" grpc.time_ms=0.649 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 346.427125] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:26.239Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"1acdec921050a4ceea82d1bf","created_at":"2026-07-25T11:31:26.004Z","trace_propagation_headers":{"sentry-trace":"304b49c0dffa41f1a9832a65ffb85956-65ff8f1dd8874741","baggage":"sentry-trace_id=304b49c0dffa41f1a9832a65ffb85956,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGPNG3K1D68V0J58NCWGWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:26.019Z","job_size_bytes":2,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-1acdec921050a4ceea82d1bf: done: 0.151664 sec","job_status":"done","queue_duration_s":0.068621,"scheduling_latency_s":0.068647,"redis_calls":14,"redis_duration_s":0.004279,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000925,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001964,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00139,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.09584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.11786312,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.151664,"completed_at":"2026-07-25T11:31:26.239Z","load_balancing_strategy":"primary","db_duration_s":0.046134,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 346.770927] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:26.583Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"cd92e28b5c5bf508058e7ed6","created_at":"2026-07-25T11:31:26.571Z","trace_propagation_headers":{"sentry-trace":"304b49c0dffa41f1a9832a65ffb85956-65ff8f1dd8874741","baggage":"sentry-trace_id=304b49c0dffa41f1a9832a65ffb85956,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGPNG3K1D68V0J58NCWGWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:26.580Z","job_size_bytes":5,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-cd92e28b5c5bf508058e7ed6: start","job_status":"start","queue_duration_s":0.002511,"scheduling_latency_s":0.002573} vm-test-run-nixbot-gitlab> gitlab # [ 346.913091] gitaly[1447]: time="2026-07-25T11:31:26.725Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPNG3K1D68V0J58NCWGWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-25T11:31:36.077" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:26.707" grpc.time_ms=16.988 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 347.004491] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:26.816Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"cd92e28b5c5bf508058e7ed6","created_at":"2026-07-25T11:31:26.571Z","trace_propagation_headers":{"sentry-trace":"304b49c0dffa41f1a9832a65ffb85956-65ff8f1dd8874741","baggage":"sentry-trace_id=304b49c0dffa41f1a9832a65ffb85956,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGPNG3K1D68V0J58NCWGWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:26.580Z","job_size_bytes":5,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-cd92e28b5c5bf508058e7ed6: done: 0.233703 sec","job_status":"done","queue_duration_s":0.002511,"scheduling_latency_s":0.002573,"redis_calls":12,"redis_duration_s":0.005714,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001095,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003846,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000773,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.186158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00173396,"duration_s":0.233703,"completed_at":"2026-07-25T11:31:26.816Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.035914,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 347.099682] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:26.819Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"70f119c1f6354489fb668639","created_at":"2026-07-25T11:31:26.633Z","trace_propagation_headers":{"sentry-trace":"304b49c0dffa41f1a9832a65ffb85956-65ff8f1dd8874741","baggage":"sentry-trace_id=304b49c0dffa41f1a9832a65ffb85956,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGPNG3K1D68V0J58NCWGWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:26.650Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-70f119c1f6354489fb668639: start","job_status":"start","queue_duration_s":0.169022,"scheduling_latency_s":0.169065} vm-test-run-nixbot-gitlab> gitlab # [ 347.397238] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:27.208Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"70f119c1f6354489fb668639","created_at":"2026-07-25T11:31:26.633Z","trace_propagation_headers":{"sentry-trace":"304b49c0dffa41f1a9832a65ffb85956-65ff8f1dd8874741","baggage":"sentry-trace_id=304b49c0dffa41f1a9832a65ffb85956,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGPNG3K1D68V0J58NCWGWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:26.650Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-70f119c1f6354489fb668639: done: 0.389365 sec","job_status":"done","queue_duration_s":0.169022,"scheduling_latency_s":0.169065,"redis_calls":11,"redis_duration_s":0.003636,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001112,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00191,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000614,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.049,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.314,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.314,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.047,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.263418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00042924,"duration_s":0.389365,"completed_at":"2026-07-25T11:31:27.208Z","load_balancing_strategy":"primary","db_duration_s":0.129541,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 347.460718] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:27.220Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"950e766d8f0cf4ae4772ecb7","created_at":"2026-07-25T11:31:26.658Z","trace_propagation_headers":{"sentry-trace":"304b49c0dffa41f1a9832a65ffb85956-65ff8f1dd8874741","baggage":"sentry-trace_id=304b49c0dffa41f1a9832a65ffb85956,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGPNG3K1D68V0J58NCWGWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:26.673Z","job_size_bytes":89,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-950e766d8f0cf4ae4772ecb7: start","job_status":"start","queue_duration_s":0.547274,"scheduling_latency_s":0.547338} vm-test-run-nixbot-gitlab> gitlab # [ 347.590430] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:27 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 2598 vm-test-run-nixbot-gitlab> gitlab # [ 347.597290] setup-gitlab-start[3635]: + mkdir -p /tmp/test-flake vm-test-run-nixbot-gitlab> gitlab # [ 347.599776] setup-gitlab-start[3635]: + cd /tmp/test-flake vm-test-run-nixbot-gitlab> gitlab # [ 347.602088] setup-gitlab-start[3635]: + git init -b master vm-test-run-nixbot-gitlab> gitlab # [ 347.604391] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:27.415Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"950e766d8f0cf4ae4772ecb7","created_at":"2026-07-25T11:31:26.658Z","trace_propagation_headers":{"sentry-trace":"304b49c0dffa41f1a9832a65ffb85956-65ff8f1dd8874741","baggage":"sentry-trace_id=304b49c0dffa41f1a9832a65ffb85956,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGPNG3K1D68V0J58NCWGWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:26.673Z","job_size_bytes":89,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-950e766d8f0cf4ae4772ecb7: done: 0.195087 sec","job_status":"done","queue_duration_s":0.547274,"scheduling_latency_s":0.547338,"redis_calls":11,"redis_duration_s":0.005432,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001262,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003237,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.021,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.129275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00133292,"duration_s":0.195087,"completed_at":"2026-07-25T11:31:27.415Z","load_balancing_strategy":"primary","db_duration_s":0.05107,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 347.699016] setup-gitlab-start[4101]: Initialized empty Git repository in /tmp/test-flake/.git/ vm-test-run-nixbot-gitlab> gitlab # [ 347.700939] setup-gitlab-start[3635]: + git config user.name test vm-test-run-nixbot-gitlab> gitlab # [ 347.702319] setup-gitlab-start[3635]: + git config user.email test@example.com vm-test-run-nixbot-gitlab> gitlab # [ 347.703828] setup-gitlab-start[3635]: + cat vm-test-run-nixbot-gitlab> gitlab # [ 347.704812] setup-gitlab-start[3635]: + git add flake.nix vm-test-run-nixbot-gitlab> gitlab # [ 347.705989] setup-gitlab-start[3635]: + git commit -m 'initial commit' vm-test-run-nixbot-gitlab> gitlab # [ 347.707549] setup-gitlab-start[4106]: [master (root-commit) 008ad05] initial commit vm-test-run-nixbot-gitlab> gitlab # [ 347.709234] setup-gitlab-start[4106]: 1 file changed, 10 insertions(+) vm-test-run-nixbot-gitlab> gitlab # [ 347.710635] setup-gitlab-start[4106]: create mode 100644 flake.nix vm-test-run-nixbot-gitlab> gitlab # [ 347.711957] setup-gitlab-start[3635]: + git remote add origin http://oauth2:glpat-APv49i6FJ9XKayZ3Zq8-4286MQp1OjEH.01.0w0qt1xta@gitlab/root/test-flake.git vm-test-run-nixbot-gitlab> gitlab # [ 347.714899] setup-gitlab-start[3635]: + git push -u origin master vm-test-run-nixbot-gitlab> gitlab # [ 347.974469] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:27 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 218 vm-test-run-nixbot-gitlab> gitlab # [ 348.362060] gitaly[1447]: time="2026-07-25T11:31:28.174Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=3 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRDC954J7MR0V8MJF4SE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:28.155" grpc.time_ms=19.36 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 348.379460] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:28 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 386 vm-test-run-nixbot-gitlab> gitlab # [ 348.830146] gitaly[1447]: time="2026-07-25T11:31:28.643Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-25T11:31:38.004" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:28.625" grpc.time_ms=17.794 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 348.905777] gitaly[1447]: time="2026-07-25T11:31:28.718Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-25T11:31:58.004" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-25T11:31:28.704" grpc.time_ms=13.96 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 348.958703] gitaly[1447]: time="2026-07-25T11:31:28.771Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB duration_ms=283ns method=POST pid=1447 status=200 url="http://unix/api/v4/internal/allowed" vm-test-run-nixbot-gitlab> gitlab # [ 349.063742] gitaly[1447]: time="2026-07-25T11:31:28.876Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB duration_ms=103ns method=POST pid=1447 status=200 url="http://unix/api/v4/internal/pre_receive" vm-test-run-nixbot-gitlab> gitlab # [ 349.077961] gitaly[1447]: time="2026-07-25T11:31:28.877Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5318 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-25T11:31:28.484" grpc.time_ms=393.43 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 349.129921] gitaly[1447]: time="2026-07-25T11:31:28.942Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4788 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-25T11:31:28.940" grpc.time_ms=1.963 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 349.215842] gitaly[1447]: time="2026-07-25T11:31:29.028Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-25T11:31:29.027" grpc.time_ms=1.061 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 349.298367] gitaly[1447]: time="2026-07-25T11:31:29.111Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4789 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-25T11:31:29.110" grpc.time_ms=0.924 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 349.473816] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:29.284Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 008ad05c6b72d3c5b6b0aa9fcd9a124deadf7afe refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"1b4405a05c7130faa2b61169","created_at":"2026-07-25T11:31:29.269Z","trace_propagation_headers":{"sentry-trace":"f7e439fbb5bd4ac789569dbe75e38afb-0644dbdd1e4a477e","baggage":"sentry-trace_id=f7e439fbb5bd4ac789569dbe75e38afb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGPRT6Q34EHQAK4KA6B2GB","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:29.281Z","job_size_bytes":185,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-1b4405a05c7130faa2b61169: start","job_status":"start","queue_duration_s":0.002777,"scheduling_latency_s":0.00285} vm-test-run-nixbot-gitlab> gitlab # [ 349.560897] gitaly[1447]: time="2026-07-25T11:31:29.372Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-25T11:31:39.003" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:29.354" grpc.time_ms=18.202 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 349.655466] gitaly[1447]: time="2026-07-25T11:31:29.468Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=30 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1437 command.oublock=0 command.real_time_ms=35 command.spawn_token_fork_ms=5 command.system_time_ms=30 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-25T11:31:39.076" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:29.427" grpc.time_ms=40.488 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 349.694731] gitaly[1447]: time="2026-07-25T11:31:29.491Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB duration_ms=298ns method=POST pid=1447 status=200 url="http://unix/api/v4/internal/post_receive" vm-test-run-nixbot-gitlab> gitlab # [ 349.700933] gitaly[1447]: time="2026-07-25T11:31:29.492Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4812 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-25T11:31:29.191" grpc.time_ms=300.433 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 349.720138] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:29.474Z","message":"changes: \"0000000000000000000000000000000000000000 008ad05c6b72d3c5b6b0aa9fcd9a124deadf7afe refs/heads/master\\n\""} vm-test-run-nixbot-gitlab> gitlab # [ 349.724976] gitaly[1447]: time="2026-07-25T11:31:29.513Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB diskcache=809f5b92-87a9-4ee2-b3ce-7f3ee12e84fa grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:28.364" pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 349.741976] setup-gitlab-start[4110]: To http://gitlab/root/test-flake.git vm-test-run-nixbot-gitlab> gitlab # [ 349.743321] setup-gitlab-start[4110]: * [new branch] master -> master vm-test-run-nixbot-gitlab> gitlab # [ 349.744751] setup-gitlab-start[4110]: branch 'master' set up to track 'origin/master'. vm-test-run-nixbot-gitlab> gitlab # [ 349.746604] systemd[1]: Finished setup-gitlab.service. vm-test-run-nixbot-gitlab> gitlab # [ 349.748119] gitlab-workhorse[1369]: time="2026-07-25T11:31:29Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 vm-test-run-nixbot-gitlab> gitlab # [ 349.750642] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:29 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 1315 vm-test-run-nixbot-gitlab> gitlab # [ 349.753315] gitaly[1447]: time="2026-07-25T11:31:29.513Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=370 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13160 command.oublock=32 command.real_time_ms=1139 command.spawn_token_fork_ms=2 command.system_time_ms=186 command.user_time_ms=184 component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:28.364" grpc.time_ms=1148.269 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 349.769876] systemd[1]: Starting nixbot CI service... vm-test-run-nixbot-gitlab> gitlab # [ 349.770695] gitaly[1447]: time="2026-07-25T11:31:29.523Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=5 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-25T11:31:39.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:29.506" grpc.time_ms=16.956 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 349.785282] gitaly[1447]: time="2026-07-25T11:31:29.546Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-25T11:31:39.034" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:29.534" grpc.time_ms=11.722 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 349.799169] gitaly[1447]: time="2026-07-25T11:31:29.600Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-25T11:31:39.086" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:29.586" grpc.time_ms=13.821 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 349.846277] gitaly[1447]: time="2026-07-25T11:31:29.659Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3938 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-25T11:31:29.658" grpc.time_ms=0.637 pid=1447 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 349.888416] gitaly[1447]: time="2026-07-25T11:31:29.701Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3940 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-25T11:31:29.696" grpc.time_ms=0.74 pid=1447 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 349.906456] gitaly[1447]: time="2026-07-25T11:31:29.707Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB diskcache=38a6be2b-236b-453b-976b-657f682313ba grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-25T11:31:39.119" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:29.620" pid=1447 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 349.922762] gitaly[1447]: time="2026-07-25T11:31:29.707Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=83 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4993 command.oublock=8 command.real_time_ms=80 command.spawn_token_fork_ms=1 command.system_time_ms=60 command.user_time_ms=23 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-25T11:31:39.119" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:29.620" grpc.time_ms=87.367 pid=1447 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 349.943367] gitaly[1447]: time="2026-07-25T11:31:29.733Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-25T11:31:39.232" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:29.732" grpc.time_ms=0.215 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 349.958966] gitaly[1447]: time="2026-07-25T11:31:29.761Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=862 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=4 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-25T11:31:39.239" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:29.740" grpc.time_ms=20.775 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 350.103848] gitaly[1447]: time="2026-07-25T11:31:29.916Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=313 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-07-25T11:31:59.007" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:29.907" grpc.time_ms=9.03 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 350.163485] gitaly[1447]: time="2026-07-25T11:31:29.976Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-25T11:31:39.005" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:29.975" grpc.time_ms=0.61 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 350.198382] gitaly[1447]: time="2026-07-25T11:31:29.992Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=316 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-25T11:31:39.011" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-07-25T11:31:29.982" grpc.time_ms=9.872 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 350.233130] gitaly[1447]: time="2026-07-25T11:31:30.033Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-25T11:31:40.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:30.023" grpc.time_ms=9.386 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 350.265474] gitaly[1447]: time="2026-07-25T11:31:30.050Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-25T11:31:40.020" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:30.040" grpc.time_ms=9.994 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 350.294791] gitaly[1447]: time="2026-07-25T11:31:30.095Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=884 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=4 command.system_time_ms=17 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-25T11:32:00.067" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:30.067" grpc.time_ms=28.062 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit setup-gitlab.service, in 351.18 seconds) vm-test-run-nixbot-gitlab> gitlab: waiting for unit nixbot.service vm-test-run-nixbot-gitlab> gitlab # [ 350.863198] gitaly[1447]: time="2026-07-25T11:31:30.675Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-25T11:31:40.172" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:30.672" grpc.time_ms=2.452 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 350.946037] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:30.757Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 008ad05c6b72d3c5b6b0aa9fcd9a124deadf7afe refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"1b4405a05c7130faa2b61169","created_at":"2026-07-25T11:31:29.269Z","trace_propagation_headers":{"sentry-trace":"f7e439fbb5bd4ac789569dbe75e38afb-0644dbdd1e4a477e","baggage":"sentry-trace_id=f7e439fbb5bd4ac789569dbe75e38afb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGPRT6Q34EHQAK4KA6B2GB","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:29.281Z","job_size_bytes":185,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-1b4405a05c7130faa2b61169: done: 1.472658 sec","job_status":"done","queue_duration_s":0.002777,"scheduling_latency_s":0.00285,"gitaly_calls":13,"gitaly_duration_s":0.302411,"redis_calls":195,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.062915,"redis_read_bytes":1461,"redis_write_bytes":18936,"redis_cache_calls":14,"redis_cache_duration_s":0.005903,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":28,"redis_feature_flag_duration_s":0.009272,"redis_feature_flag_read_bytes":1044,"redis_feature_flag_write_bytes":3487,"redis_queues_calls":8,"redis_queues_duration_s":0.001797,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4934,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.006154,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.022939,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.01685,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.229,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.306,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.344,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.229,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.867108,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00075856,"exclusive_lock_hold_duration_s":0.045494940000000005,"duration_s":1.472658,"completed_at":"2026-07-25T11:31:30.756Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.282791,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 351.094240] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:30.760Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"ce88d3db1cc95ef5b7e1c837","created_at":"2026-07-25T11:31:29.549Z","trace_propagation_headers":{"sentry-trace":"f7e439fbb5bd4ac789569dbe75e38afb-bc86368d5ea74e1a","baggage":"sentry-trace_id=f7e439fbb5bd4ac789569dbe75e38afb,sentry-release=Unknown"},"correlation_id":"01KYCGPRT6Q34EHQAK4KA6B2GB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:29.560Z","job_size_bytes":46,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-ce88d3db1cc95ef5b7e1c837: start","job_status":"start","queue_duration_s":1.200281,"scheduling_latency_s":1.200324} vm-test-run-nixbot-gitlab> gitlab # [ 351.125087] nixbot[4200]: {"ts": "2026-07-25T11:31:30.905Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} vm-test-run-nixbot-gitlab> gitlab # [ 351.128777] nixbot[4200]: {"ts": "2026-07-25T11:31:30.930Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} vm-test-run-nixbot-gitlab> gitlab # [ 351.132206] gitaly[1447]: time="2026-07-25T11:31:30.885Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-07-25T17:31:30.873" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:30.874" grpc.time_ms=9.61 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 351.149115] gitaly[1447]: time="2026-07-25T11:31:30.925Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=592 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-25T11:32:00.004" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:30.905" grpc.time_ms=19.555 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 351.169241] nixbot[4200]: {"ts": "2026-07-25T11:31:30.949Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} vm-test-run-nixbot-gitlab> gitlab # [ 351.172245] nixbot[4200]: {"ts": "2026-07-25T11:31:30.957Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} vm-test-run-nixbot-gitlab> gitlab # [ 351.175290] nixbot[4200]: {"ts": "2026-07-25T11:31:30.961Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} vm-test-run-nixbot-gitlab> gitlab # [ 351.178206] nixbot[4200]: {"ts": "2026-07-25T11:31:30.966Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} vm-test-run-nixbot-gitlab> gitlab # [ 351.181247] nixbot[4200]: {"ts": "2026-07-25T11:31:30.970Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} vm-test-run-nixbot-gitlab> gitlab # [ 351.184107] nixbot[4200]: {"ts": "2026-07-25T11:31:30.972Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} vm-test-run-nixbot-gitlab> gitlab # [ 351.186950] nixbot[4200]: {"ts": "2026-07-25T11:31:30.976Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} vm-test-run-nixbot-gitlab> gitlab # [ 351.189864] nixbot[4200]: {"ts": "2026-07-25T11:31:30.978Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} vm-test-run-nixbot-gitlab> gitlab # [ 351.192916] nixbot[4200]: {"ts": "2026-07-25T11:31:30.983Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} vm-test-run-nixbot-gitlab> gitlab # [ 351.195659] nixbot[4200]: {"ts": "2026-07-25T11:31:30.989Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} vm-test-run-nixbot-gitlab> gitlab # [ 351.198554] nixbot[4200]: {"ts": "2026-07-25T11:31:30.992Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} vm-test-run-nixbot-gitlab> gitlab # [ 351.201540] nixbot[4200]: {"ts": "2026-07-25T11:31:30.993Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} vm-test-run-nixbot-gitlab> gitlab # [ 351.204387] nixbot[4200]: {"ts": "2026-07-25T11:31:30.996Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} vm-test-run-nixbot-gitlab> gitlab # [ 351.207030] nixbot[4200]: {"ts": "2026-07-25T11:31:30.998Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} vm-test-run-nixbot-gitlab> gitlab # [ 351.209626] nixbot[4200]: {"ts": "2026-07-25T11:31:30.999Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} vm-test-run-nixbot-gitlab> gitlab # [ 351.212038] nixbot[4200]: {"ts": "2026-07-25T11:31:31.001Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} vm-test-run-nixbot-gitlab> gitlab # [ 351.214323] nixbot[4200]: {"ts": "2026-07-25T11:31:31.004Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} vm-test-run-nixbot-gitlab> gitlab # [ 351.217100] nixbot[4200]: {"ts": "2026-07-25T11:31:31.005Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} vm-test-run-nixbot-gitlab> gitlab # [ 351.219517] nixbot[4200]: {"ts": "2026-07-25T11:31:31.006Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} vm-test-run-nixbot-gitlab> gitlab # [ 351.222062] nixbot[4200]: {"ts": "2026-07-25T11:31:31.010Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} vm-test-run-nixbot-gitlab> gitlab # [ 351.225641] nixbot[4200]: {"ts": "2026-07-25T11:31:31.012Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} vm-test-run-nixbot-gitlab> gitlab # [ 351.229642] nixbot[4200]: {"ts": "2026-07-25T11:31:31.014Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} vm-test-run-nixbot-gitlab> gitlab # [ 351.231942] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:31.013Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"ce88d3db1cc95ef5b7e1c837","created_at":"2026-07-25T11:31:29.549Z","trace_propagation_headers":{"sentry-trace":"f7e439fbb5bd4ac789569dbe75e38afb-bc86368d5ea74e1a","baggage":"sentry-trace_id=f7e439fbb5bd4ac789569dbe75e38afb,sentry-release=Unknown"},"correlation_id":"01KYCGPRT6Q34EHQAK4KA6B2GB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:29.560Z","job_size_bytes":46,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-ce88d3db1cc95ef5b7e1c837: done: 0.252863 sec","job_status":"done","queue_duration_s":1.200281,"scheduling_latency_s":1.200324,"gitaly_calls":2,"gitaly_duration_s":0.038952,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.017807,"redis_read_bytes":68,"redis_write_bytes":4887,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.005518,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.001776,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002911,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.006156,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":752,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001446,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.009,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.169769,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00074484,"duration_s":0.252863,"completed_at":"2026-07-25T11:31:31.013Z","load_balancing_strategy":"primary","db_duration_s":0.027895,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 351.284922] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:31.017Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"a9cc619d308502c08089232e","created_at":"2026-07-25T11:31:29.574Z","trace_propagation_headers":{"sentry-trace":"f7e439fbb5bd4ac789569dbe75e38afb-bc86368d5ea74e1a","baggage":"sentry-trace_id=f7e439fbb5bd4ac789569dbe75e38afb,sentry-release=Unknown"},"correlation_id":"01KYCGPRT6Q34EHQAK4KA6B2GB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:29.581Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-a9cc619d308502c08089232e: start","job_status":"start","queue_duration_s":1.436323,"scheduling_latency_s":1.436377} vm-test-run-nixbot-gitlab> gitlab # [ 351.314936] gitaly[1447]: time="2026-07-25T11:31:31.128Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB diskcache=a54d91ab-5be7-4191-adf2-7862ef6e02ae grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-25T17:31:31.094" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:31.095" pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 351.326637] gitaly[1447]: time="2026-07-25T11:31:31.128Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1175 command.oublock=0 command.real_time_ms=39 command.spawn_token_fork_ms=8 command.system_time_ms=10 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-25T17:31:31.094" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:31.095" grpc.time_ms=32.928 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 351.407083] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:31.219Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"a9cc619d308502c08089232e","created_at":"2026-07-25T11:31:29.574Z","trace_propagation_headers":{"sentry-trace":"f7e439fbb5bd4ac789569dbe75e38afb-bc86368d5ea74e1a","baggage":"sentry-trace_id=f7e439fbb5bd4ac789569dbe75e38afb,sentry-release=Unknown"},"correlation_id":"01KYCGPRT6Q34EHQAK4KA6B2GB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:29.581Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-a9cc619d308502c08089232e: done: 0.201554 sec","job_status":"done","queue_duration_s":1.436323,"scheduling_latency_s":1.436377,"gitaly_calls":1,"gitaly_duration_s":0.037577,"redis_calls":15,"redis_duration_s":0.013464,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000657,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009446,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001031,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00233,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.022,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.042,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.11535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.12330758,"duration_s":0.201554,"completed_at":"2026-07-25T11:31:31.219Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02732,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 351.493386] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:31.222Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"098c91a6295bc71b4bed64ad","created_at":"2026-07-25T11:31:29.725Z","trace_propagation_headers":{"sentry-trace":"f7e439fbb5bd4ac789569dbe75e38afb-bc86368d5ea74e1a","baggage":"sentry-trace_id=f7e439fbb5bd4ac789569dbe75e38afb,sentry-release=Unknown"},"correlation_id":"01KYCGPRT6Q34EHQAK4KA6B2GB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:29.728Z","job_size_bytes":23,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-098c91a6295bc71b4bed64ad: start","job_status":"start","queue_duration_s":1.493635,"scheduling_latency_s":1.493679} vm-test-run-nixbot-gitlab> gitlab # [ 351.511113] gitaly[1447]: time="2026-07-25T11:31:31.293Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=593 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-25T11:32:01.275" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:31.275" grpc.time_ms=17.267 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 351.525070] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:31.318Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"098c91a6295bc71b4bed64ad","created_at":"2026-07-25T11:31:29.725Z","trace_propagation_headers":{"sentry-trace":"f7e439fbb5bd4ac789569dbe75e38afb-bc86368d5ea74e1a","baggage":"sentry-trace_id=f7e439fbb5bd4ac789569dbe75e38afb,sentry-release=Unknown"},"correlation_id":"01KYCGPRT6Q34EHQAK4KA6B2GB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:29.728Z","job_size_bytes":23,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-098c91a6295bc71b4bed64ad: done: 0.096183 sec","job_status":"done","queue_duration_s":1.493635,"scheduling_latency_s":1.493679,"gitaly_calls":1,"gitaly_duration_s":0.020432,"redis_calls":16,"redis_duration_s":0.008277,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000513,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001927,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002459,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.003378,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059537,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.096183,"completed_at":"2026-07-25T11:31:31.318Z","load_balancing_strategy":"primary","db_duration_s":0.005949,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 351.570893] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:31.321Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"33ca69db745310698866749b","created_at":"2026-07-25T11:31:30.597Z","trace_propagation_headers":{"sentry-trace":"f7e439fbb5bd4ac789569dbe75e38afb-bc86368d5ea74e1a","baggage":"sentry-trace_id=f7e439fbb5bd4ac789569dbe75e38afb,sentry-release=Unknown"},"correlation_id":"01KYCGPRT6Q34EHQAK4KA6B2GB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d1b5dca3d63e188152155730c4a2d81c93c14a6e386bd574be84eae2ea62a0d6","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:30.608Z","job_size_bytes":467,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-33ca69db745310698866749b: start","job_status":"start","queue_duration_s":0.713163,"scheduling_latency_s":0.713207} vm-test-run-nixbot-gitlab> gitlab # [ 351.611271] gitaly[1447]: time="2026-07-25T11:31:31.424Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:01.422" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:31.422" grpc.time_ms=1.231 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 351.648570] gitaly[1447]: time="2026-07-25T11:31:31.443Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-25T11:31:41.009" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:31.430" grpc.time_ms=12.165 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 351.698805] nixbot[4200]: INFO: Started server process [4200] vm-test-run-nixbot-gitlab> gitlab # [ 351.701440] nixbot[4200]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) vm-test-run-nixbot-gitlab> gitlab # [ 351.707595] nixbot[4200]: INFO: ::ffff:127.0.0.1:51702 - "GET /health HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab # [ 351.716415] systemd[1]: Started nixbot CI service. vm-test-run-nixbot-gitlab> gitlab # [ 351.720790] systemd[1]: Reached target Multi-User System. vm-test-run-nixbot-gitlab> gitlab # [ 351.722518] systemd[1]: Startup finished in 765ms (kernel) + 5.130s (initrd) + 5min 45.825s (userspace) = 5min 51.722s. vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit nixbot.service, in 1.13 seconds) vm-test-run-nixbot-gitlab> gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health vm-test-run-nixbot-gitlab> gitlab # [ 351.870884] gitaly[1447]: time="2026-07-25T11:31:31.683Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-25T11:32:01.681" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:31.681" grpc.time_ms=1.232 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 351.891689] nixbot[4200]: INFO: ::ffff:127.0.0.1:52154 - "GET /health HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.06 seconds) vm-test-run-nixbot-gitlab> subtest: project discovered and webhook registered vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks vm-test-run-nixbot-gitlab> gitlab # [ 351.900132] gitaly[1447]: time="2026-07-25T11:31:31.713Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=236 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-25T11:32:01.707" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:31.707" grpc.time_ms=5.344 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 352.046901] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 93 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.16 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 352.095738] gitaly[1447]: time="2026-07-25T11:31:31.908Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-25T11:31:41.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:31.893" grpc.time_ms=15.276 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 352.564829] gitaly[1447]: time="2026-07-25T11:31:32.377Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=11 component=gitaly.StreamServerInterceptor correlation_id=01KYCGPW1YJQ8MCKN6G48KY07C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-25T11:32:02.049" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:32.349" grpc.time_ms=28.176 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.13 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 353.176733] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:32 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 64 vm-test-run-nixbot-gitlab> gitlab # [ 353.215753] gitaly[1447]: time="2026-07-25T11:31:33.027Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=883 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-25T11:32:03.003" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:33.004" grpc.time_ms=23.652 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 353.259203] gitaly[1447]: time="2026-07-25T11:31:33.045Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGPRT6Q34EHQAK4KA6B2GB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-25T11:32:03.031" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:33.031" grpc.time_ms=13.228 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 353.294758] sidekiq-cluster[2673]: {"severity":"WARN","time":"2026-07-25T11:31:33.106Z","class":"CreatePipelineWorker","correlation_id":"01KYCGPRT6Q34EHQAK4KA6B2GB","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"008ad05c6b72d3c5b6b0aa9fcd9a124deadf7afe","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"008ad05c6b72d3c5b6b0aa9fcd9a124deadf7afe","ref":"refs/heads/master"},"retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 353.310902] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:33.121Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"33ca69db745310698866749b","created_at":"2026-07-25T11:31:30.597Z","trace_propagation_headers":{"sentry-trace":"f7e439fbb5bd4ac789569dbe75e38afb-bc86368d5ea74e1a","baggage":"sentry-trace_id=f7e439fbb5bd4ac789569dbe75e38afb,sentry-release=Unknown"},"correlation_id":"01KYCGPRT6Q34EHQAK4KA6B2GB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d1b5dca3d63e188152155730c4a2d81c93c14a6e386bd574be84eae2ea62a0d6","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:30.608Z","job_size_bytes":467,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-33ca69db745310698866749b: done: 1.800166 sec","job_status":"done","queue_duration_s":0.713163,"scheduling_latency_s":0.713207,"gitaly_calls":7,"gitaly_duration_s":0.110247,"redis_calls":53,"redis_duration_s":0.020322,"redis_read_bytes":171,"redis_write_bytes":4427,"redis_cache_calls":4,"redis_cache_duration_s":0.001746,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.009881,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002037,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.004902,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001756,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":40,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":40,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.073,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.063,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.371681,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00060626,"duration_s":1.800166,"completed_at":"2026-07-25T11:31:33.121Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.143695,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 353.383164] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:33.126Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"797b2447eb9bff54c9627f5e","created_at":"2026-07-25T11:31:30.957Z","trace_propagation_headers":{"sentry-trace":"f7e439fbb5bd4ac789569dbe75e38afb-922d668a2ec24989","baggage":"sentry-trace_id=f7e439fbb5bd4ac789569dbe75e38afb,sentry-release=Unknown"},"correlation_id":"01KYCGPRT6Q34EHQAK4KA6B2GB","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:30.973Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-797b2447eb9bff54c9627f5e: start","job_status":"start","queue_duration_s":2.152473,"scheduling_latency_s":2.152521} vm-test-run-nixbot-gitlab> gitlab # [ 353.408494] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:33.220Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"797b2447eb9bff54c9627f5e","created_at":"2026-07-25T11:31:30.957Z","trace_propagation_headers":{"sentry-trace":"f7e439fbb5bd4ac789569dbe75e38afb-922d668a2ec24989","baggage":"sentry-trace_id=f7e439fbb5bd4ac789569dbe75e38afb,sentry-release=Unknown"},"correlation_id":"01KYCGPRT6Q34EHQAK4KA6B2GB","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:30.973Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-797b2447eb9bff54c9627f5e: done: 0.093767 sec","job_status":"done","queue_duration_s":2.152473,"scheduling_latency_s":2.152521,"redis_calls":19,"redis_duration_s":0.00701,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001539,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000914,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.00316,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001397,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067485,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01738668,"duration_s":0.093767,"completed_at":"2026-07-25T11:31:33.219Z","load_balancing_strategy":"primary","db_duration_s":0.014806,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 353.460231] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:33.224Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"8af825a514610640de8a2e15","created_at":"2026-07-25T11:31:33.192Z","trace_propagation_headers":{"sentry-trace":"f7e439fbb5bd4ac789569dbe75e38afb-b845f46a469544f9","baggage":"sentry-trace_id=f7e439fbb5bd4ac789569dbe75e38afb,sentry-release=Unknown"},"correlation_id":"01KYCGPRT6Q34EHQAK4KA6B2GB","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:33.204Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-8af825a514610640de8a2e15: start","job_status":"start","queue_duration_s":0.019929,"scheduling_latency_s":0.019964} vm-test-run-nixbot-gitlab> gitlab # [ 353.558811] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:33.371Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"8af825a514610640de8a2e15","created_at":"2026-07-25T11:31:33.192Z","trace_propagation_headers":{"sentry-trace":"f7e439fbb5bd4ac789569dbe75e38afb-b845f46a469544f9","baggage":"sentry-trace_id=f7e439fbb5bd4ac789569dbe75e38afb,sentry-release=Unknown"},"correlation_id":"01KYCGPRT6Q34EHQAK4KA6B2GB","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:33.204Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-8af825a514610640de8a2e15: done: 0.147387 sec","job_status":"done","queue_duration_s":0.019929,"scheduling_latency_s":0.019964,"redis_calls":11,"redis_duration_s":0.003719,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000737,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00234,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000642,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.011,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.103345,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004367,"duration_s":0.147387,"completed_at":"2026-07-25T11:31:33.371Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.038471,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks vm-test-run-nixbot-gitlab> gitlab # [ 354.343990] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 99 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.17 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 355.118866] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:34 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 3412 vm-test-run-nixbot-gitlab> gitlab # [ 355.120857] nixbot[4200]: {"ts": "2026-07-25T11:31:34.933Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 355.141899] nixbot[4200]: {"ts": "2026-07-25T11:31:34.954Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} vm-test-run-nixbot-gitlab> gitlab # [ 355.228064] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 75 vm-test-run-nixbot-gitlab> gitlab # [ 355.234817] nixbot[4200]: {"ts": "2026-07-25T11:31:35.042Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 355.241942] nixbot[4200]: {"ts": "2026-07-25T11:31:35.045Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks vm-test-run-nixbot-gitlab> gitlab # [ 355.394288] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:35 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 157 vm-test-run-nixbot-gitlab> gitlab # [ 355.404579] nixbot[4200]: {"ts": "2026-07-25T11:31:35.207Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.20 seconds) vm-test-run-nixbot-gitlab> (finished: subtest: project discovered and webhook registered, in 3.66 seconds) vm-test-run-nixbot-gitlab> subtest: push delivers the webhook vm-test-run-nixbot-gitlab> gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master vm-test-run-nixbot-gitlab> gitlab # [ 355.551584] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 153 vm-test-run-nixbot-gitlab> gitlab # [ 355.592284] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:35 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 184 vm-test-run-nixbot-gitlab> gitlab # [ 355.595372] nixbot[4200]: {"ts": "2026-07-25T11:31:35.403Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 355.643219] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:35 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 37 vm-test-run-nixbot-gitlab> gitlab # [ 355.690202] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 83 vm-test-run-nixbot-gitlab> gitlab # [ 355.693248] nixbot[4200]: {"ts": "2026-07-25T11:31:35.504Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 355.751880] gitaly[1447]: time="2026-07-25T11:31:35.564Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGPZWZBS7K1V9SMRQ8YWYA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:35.549" grpc.time_ms=14.879 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 355.766545] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:35 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.54.0" 113 vm-test-run-nixbot-gitlab> gitlab # Enumerating objects: 5, done. vm-test-run-nixbot-gitlab> gitlab # [ 355.768950] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:35 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 71 vm-test-run-nixbot-gitlab> gitlab # Counting objects: 20% (1/5)[ 355.771350] nixbot[4200]: {"ts": "2026-07-25T11:31:35.580Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. vm-test-run-nixbot-gitlab> gitlab # Delta compression using up to 4 threads vm-test-run-nixbot-gitlab> gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. vm-test-run-nixbot-gitlab> gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 272 bytes | 272.00 KiB/s, done. vm-test-run-nixbot-gitlab> gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) vm-test-run-nixbot-gitlab> gitlab # [ 355.837928] gitaly[1447]: time="2026-07-25T11:31:35.651Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ0114831MWTB08MRHJJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-07-25T11:32:05.340" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:35.641" grpc.time_ms=9.54 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 355.864313] gitaly[1447]: time="2026-07-25T11:31:35.671Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ0114831MWTB08MRHJJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:05.370" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:35.670" grpc.time_ms=0.646 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 355.878900] gitaly[1447]: time="2026-07-25T11:31:35.684Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ0114831MWTB08MRHJJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-25T11:31:45.005" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:35.676" grpc.time_ms=8.446 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 356.012499] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 239 vm-test-run-nixbot-gitlab> gitlab # [ 356.015041] nixbot[4200]: {"ts": "2026-07-25T11:31:35.826Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 356.024148] gitaly[1447]: time="2026-07-25T11:31:35.828Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=592 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-25T11:31:45.197" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:35.817" grpc.time_ms=10.224 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 356.072595] gitaly[1447]: time="2026-07-25T11:31:35.885Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=7 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-25T11:32:05.170" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-25T11:31:35.870" grpc.time_ms=15.232 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 356.104134] gitaly[1447]: time="2026-07-25T11:31:35.903Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE duration_ms=145ns method=POST pid=1447 status=200 url="http://unix/api/v4/internal/allowed" vm-test-run-nixbot-gitlab> gitlab # [ 356.110108] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 93 vm-test-run-nixbot-gitlab> gitlab # [ 356.112806] nixbot[4200]: {"ts": "2026-07-25T11:31:35.924Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 356.115451] nixbot[4200]: {"ts": "2026-07-25T11:31:35.928Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} vm-test-run-nixbot-gitlab> gitlab # [ 356.123339] gitaly[1447]: time="2026-07-25T11:31:35.936Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE duration_ms=32ns method=POST pid=1447 status=200 url="http://unix/api/v4/internal/pre_receive" vm-test-run-nixbot-gitlab> gitlab # [ 356.126499] gitaly[1447]: time="2026-07-25T11:31:35.937Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5318 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-25T11:31:35.757" grpc.time_ms=179.859 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 356.149464] gitaly[1447]: time="2026-07-25T11:31:35.962Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4788 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-25T11:31:35.961" grpc.time_ms=0.804 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 356.183659] gitaly[1447]: time="2026-07-25T11:31:35.996Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-25T11:31:35.995" grpc.time_ms=0.84 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 356.221091] gitaly[1447]: time="2026-07-25T11:31:36.031Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4789 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-25T11:31:36.029" grpc.time_ms=1.244 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 356.237924] gitaly[1447]: time="2026-07-25T11:31:36.049Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=11 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ0D2JY1RT60TTCAEG6XC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:36.028" grpc.time_ms=21.204 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 356.252377] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:36 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 80 vm-test-run-nixbot-gitlab> gitlab # [ 356.348755] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:36.155Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","008ad05c6b72d3c5b6b0aa9fcd9a124deadf7afe 36ae1414ecb17a3920be4f602d1012d76d4248c9 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"2ef0596bb9824e716f900886","created_at":"2026-07-25T11:31:36.115Z","trace_propagation_headers":{"sentry-trace":"fc8c5dc0b26f4fdf89361d7b7289c9af-d48737140c434525","baggage":"sentry-trace_id=fc8c5dc0b26f4fdf89361d7b7289c9af,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGQ01FCE950ZMHF2RYKPNE","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:36.148Z","job_size_bytes":185,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-2ef0596bb9824e716f900886: start","job_status":"start","queue_duration_s":0.007403,"scheduling_latency_s":0.007471} vm-test-run-nixbot-gitlab> gitlab # [ 356.367945] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:36.179Z","message":"changes: \"008ad05c6b72d3c5b6b0aa9fcd9a124deadf7afe 36ae1414ecb17a3920be4f602d1012d76d4248c9 refs/heads/master\\n\""} vm-test-run-nixbot-gitlab> gitlab # [ 356.370730] gitlab-workhorse[1369]: time="2026-07-25T11:31:36Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=149 vm-test-run-nixbot-gitlab> gitlab # [ 356.372684] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:36 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 114 vm-test-run-nixbot-gitlab> gitlab # [ 356.374615] gitaly[1447]: time="2026-07-25T11:31:36.165Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ0FKPP9CJMAPSF729F6B grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:36.146" pid=1447 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 356.385183] gitaly[1447]: time="2026-07-25T11:31:36.165Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ0FKPP9CJMAPSF729F6B grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:36.146" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 356.405068] gitaly[1447]: time="2026-07-25T11:31:36.165Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ0FKPP9CJMAPSF729F6B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:36.146" grpc.time_ms=19.086 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 356.421923] gitaly[1447]: time="2026-07-25T11:31:36.220Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=12 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-25T11:31:46.182" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:36.202" grpc.time_ms=16.929 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 356.436530] gitaly[1447]: time="2026-07-25T11:31:36.232Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-25T11:31:46.205" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:36.225" grpc.time_ms=6.576 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 356.452162] gitaly[1447]: time="2026-07-25T11:31:36.260Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-25T11:31:46.232" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:36.252" grpc.time_ms=7.557 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 356.467572] gitaly[1447]: time="2026-07-25T11:31:36.277Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=4 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=4 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=598 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-07-25T11:32:06.264" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:36.265" grpc.time_ms=12.706 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 356.489237] gitaly[1447]: time="2026-07-25T11:31:36.302Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=319 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-25T11:31:46.276" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-07-25T11:31:36.296" grpc.time_ms=5.411 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 356.503092] gitaly[1447]: time="2026-07-25T11:31:36.309Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE duration_ms=233ns method=POST pid=1447 status=200 url="http://unix/api/v4/internal/post_receive" vm-test-run-nixbot-gitlab> gitlab # [ 356.509945] gitaly[1447]: time="2026-07-25T11:31:36.317Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4812 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-25T11:31:36.074" grpc.time_ms=242.46 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 356.524992] gitaly[1447]: time="2026-07-25T11:31:36.338Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE diskcache=dc0c4c39-51b8-4d53-a99a-335d9d8a1f89 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:35.704" pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 356.536571] gitaly[1447]: time="2026-07-25T11:31:36.341Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=7 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-25T11:31:46.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:36.328" grpc.time_ms=12.818 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 356.550397] gitlab-workhorse[1369]: time="2026-07-25T11:31:36Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 vm-test-run-nixbot-gitlab> gitlab # To http://gitlab/root/test-flake.git vm-test-run-nixbot-gitlab> gitlab # [ 356.552345] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:36 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 749 vm-test-run-nixbot-gitlab> gitlab # 008ad05..36ae141 [ 356.554679] gitaly[1447]: time="2026-07-25T11:31:36.348Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=139 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13181 command.oublock=32 command.real_time_ms=616 command.spawn_token_fork_ms=1 command.system_time_ms=85 command.user_time_ms=54 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=604 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:35.704" grpc.time_ms=644.05 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # master -> master vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.04 seconds) vm-test-run-nixbot-gitlab> gitlab: must succeed: git -C /tmp/test-flake rev-parse master vm-test-run-nixbot-gitlab> gitlab # [ 356.596115] gitaly[1447]: time="2026-07-25T11:31:36.408Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ0KD1J0JSAM475WPFGCT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:36.391" pid=1447 remote_ip="2001:db8:1::1" request_sha=7ceffe2e3bfecf6f29d65bafeb6cb6fc58c564c7 response_bytes=417 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 356.606877] gitlab-workhorse[1369]: time="2026-07-25T11:31:36Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=417 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 356.608961] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:36 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 417 "" "git/2.54.0" 235 vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=36ae1414ecb17a3920be4f602d1012d76d4248c9' vm-test-run-nixbot-gitlab> gitlab # [ 356.611310] gitaly[1447]: time="2026-07-25T11:31:36.408Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1133 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ0KD1J0JSAM475WPFGCT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:36.391" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=36ae1414ecb17a3920be4f602d1012d76d4248c9', in 0.03 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 356.626034] gitaly[1447]: time="2026-07-25T11:31:36.408Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1133 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ0KD1J0JSAM475WPFGCT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:36.391" grpc.time_ms=16.802 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 356.641887] nixbot[4200]: INFO: ::ffff:127.0.0.1:52168 - "GET /api/repos/gitlab/root/test-flake/builds?commit=36ae1414ecb17a3920be4f602d1012d76d4248c9 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab # [ 356.718626] gitaly[1447]: time="2026-07-25T11:31:36.531Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ0WEHT3CSXEWZB4CEDES grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:36.521" grpc.time_ms=9.192 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 356.744603] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:36 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 70 vm-test-run-nixbot-gitlab> gitlab # [ 356.773548] gitaly[1447]: time="2026-07-25T11:31:36.585Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-25T11:31:46.163" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:36.583" grpc.time_ms=2.336 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 356.803468] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:36.612Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","008ad05c6b72d3c5b6b0aa9fcd9a124deadf7afe 36ae1414ecb17a3920be4f602d1012d76d4248c9 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"2ef0596bb9824e716f900886","created_at":"2026-07-25T11:31:36.115Z","trace_propagation_headers":{"sentry-trace":"fc8c5dc0b26f4fdf89361d7b7289c9af-d48737140c434525","baggage":"sentry-trace_id=fc8c5dc0b26f4fdf89361d7b7289c9af,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGQ01FCE950ZMHF2RYKPNE","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-25T11:31:36.148Z","job_size_bytes":185,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-2ef0596bb9824e716f900886: done: 0.457096 sec","job_status":"done","queue_duration_s":0.007403,"scheduling_latency_s":0.007471,"gitaly_calls":7,"gitaly_duration_s":0.107744,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.022427,"redis_read_bytes":350,"redis_write_bytes":13067,"redis_cache_calls":9,"redis_cache_duration_s":0.002628,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000539,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.001256,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6767,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.004716,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.005324,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.007964,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.024,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.024,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.035,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.297019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00095346,"exclusive_lock_hold_duration_s":0.02948192,"duration_s":0.457096,"completed_at":"2026-07-25T11:31:36.612Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.04251,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 356.874496] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:36.614Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"8e2ef1ab5e58ca3ff10e24ca","created_at":"2026-07-25T11:31:36.235Z","trace_propagation_headers":{"sentry-trace":"fc8c5dc0b26f4fdf89361d7b7289c9af-094e23b3b345449d","baggage":"sentry-trace_id=fc8c5dc0b26f4fdf89361d7b7289c9af,sentry-release=Unknown"},"correlation_id":"01KYCGQ01FCE950ZMHF2RYKPNE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:36.238Z","job_size_bytes":46,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8e2ef1ab5e58ca3ff10e24ca: start","job_status":"start","queue_duration_s":0.376076,"scheduling_latency_s":0.376109} vm-test-run-nixbot-gitlab> gitlab # [ 356.891929] gitaly[1447]: time="2026-07-25T11:31:36.647Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ0YYY28KWGYG05H1JAS0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:36.636" pid=1447 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 356.902538] gitlab-workhorse[1369]: time="2026-07-25T11:31:36Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 356.904463] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:36 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 105 vm-test-run-nixbot-gitlab> gitlab # [ 356.906412] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:36.645Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"8e2ef1ab5e58ca3ff10e24ca","created_at":"2026-07-25T11:31:36.235Z","trace_propagation_headers":{"sentry-trace":"fc8c5dc0b26f4fdf89361d7b7289c9af-094e23b3b345449d","baggage":"sentry-trace_id=fc8c5dc0b26f4fdf89361d7b7289c9af,sentry-release=Unknown"},"correlation_id":"01KYCGQ01FCE950ZMHF2RYKPNE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:36.238Z","job_size_bytes":46,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8e2ef1ab5e58ca3ff10e24ca: done: 0.031042 sec","job_status":"done","queue_duration_s":0.376076,"scheduling_latency_s":0.376109,"redis_calls":9,"redis_duration_s":0.004366,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003161,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000239,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000966,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031042,"completed_at":"2026-07-25T11:31:36.645Z","load_balancing_strategy":"primary","db_duration_s":0.001469,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 356.950518] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:36.647Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"05a785edefb26cb4558be94f","created_at":"2026-07-25T11:31:36.245Z","trace_propagation_headers":{"sentry-trace":"fc8c5dc0b26f4fdf89361d7b7289c9af-094e23b3b345449d","baggage":"sentry-trace_id=fc8c5dc0b26f4fdf89361d7b7289c9af,sentry-release=Unknown"},"correlation_id":"01KYCGQ01FCE950ZMHF2RYKPNE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:36.248Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-05a785edefb26cb4558be94f: start","job_status":"start","queue_duration_s":0.398933,"scheduling_latency_s":0.398957} vm-test-run-nixbot-gitlab> gitlab # [ 356.967366] gitaly[1447]: time="2026-07-25T11:31:36.647Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=5 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ0YYY28KWGYG05H1JAS0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:36.636" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 356.982371] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:36 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 66 vm-test-run-nixbot-gitlab> gitlab # [ 356.984530] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:36.721Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"05a785edefb26cb4558be94f","created_at":"2026-07-25T11:31:36.245Z","trace_propagation_headers":{"sentry-trace":"fc8c5dc0b26f4fdf89361d7b7289c9af-094e23b3b345449d","baggage":"sentry-trace_id=fc8c5dc0b26f4fdf89361d7b7289c9af,sentry-release=Unknown"},"correlation_id":"01KYCGQ01FCE950ZMHF2RYKPNE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:36.248Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-05a785edefb26cb4558be94f: done: 0.07428 sec","job_status":"done","queue_duration_s":0.398933,"scheduling_latency_s":0.398957,"gitaly_calls":1,"gitaly_duration_s":0.031705,"redis_calls":12,"redis_duration_s":0.002351,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000516,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000942,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032893,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.05162578,"duration_s":0.07428,"completed_at":"2026-07-25T11:31:36.721Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005562,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 357.030534] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:36.723Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"d8d62198080eb6c9dc83cca7","created_at":"2026-07-25T11:31:36.394Z","trace_propagation_headers":{"sentry-trace":"fc8c5dc0b26f4fdf89361d7b7289c9af-094e23b3b345449d","baggage":"sentry-trace_id=fc8c5dc0b26f4fdf89361d7b7289c9af,sentry-release=Unknown"},"correlation_id":"01KYCGQ01FCE950ZMHF2RYKPNE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:36fb598e8d54ab2600b07e719e39dcea1bce6dad67a1d179d892e53a4478149a","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:36.399Z","job_size_bytes":467,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-d8d62198080eb6c9dc83cca7: start","job_status":"start","queue_duration_s":0.323114,"scheduling_latency_s":0.323139} vm-test-run-nixbot-gitlab> gitlab # [ 357.048352] gitaly[1447]: time="2026-07-25T11:31:36.647Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=5 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ0YYY28KWGYG05H1JAS0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:36.636" grpc.time_ms=10.817 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 357.064148] gitlab-workhorse[1369]: time="2026-07-25T11:31:36Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=214 vm-test-run-nixbot-gitlab> gitlab # [ 357.065980] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:36 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 214 "" "git/2.54.0" 105 vm-test-run-nixbot-gitlab> gitlab # [ 357.067838] gitaly[1447]: time="2026-07-25T11:31:36.699Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE diskcache=8baf7c40-9779-4b9f-aab1-fbe99f9708ba grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-25T17:31:36.669" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:36.670" pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 357.078056] gitaly[1447]: time="2026-07-25T11:31:36.699Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1511 command.oublock=0 command.real_time_ms=31 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-25T17:31:36.669" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:36.670" grpc.time_ms=29.021 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 357.093627] gitaly[1447]: time="2026-07-25T11:31:36.748Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ13A6315E5WSCXEN8DXW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:36.739" grpc.time_ms=9.128 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 357.107067] gitaly[1447]: time="2026-07-25T11:31:36.757Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:06.756" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:36.757" grpc.time_ms=0.678 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 357.119876] gitaly[1447]: time="2026-07-25T11:31:36.769Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=7 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-25T11:31:46.010" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:36.760" grpc.time_ms=8.97 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 357.133313] gitaly[1447]: time="2026-07-25T11:31:36.819Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-25T11:32:06.018" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:36.818" grpc.time_ms=1.035 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 357.146292] gitaly[1447]: time="2026-07-25T11:31:36.833Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=295 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=0 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-25T11:32:06.022" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:36.823" grpc.time_ms=10.326 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 357.162750] gitaly[1447]: time="2026-07-25T11:31:36.856Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ15FTTPD8VPH4KPCQ1G2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:36.833" pid=1447 remote_ip="2001:db8:1::1" request_sha=69dcef142412daaffeeda28f6446e517599d516e response_bytes=214 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 357.173186] gitaly[1447]: time="2026-07-25T11:31:36.856Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=12 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ15FTTPD8VPH4KPCQ1G2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:36.833" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 357.188339] gitaly[1447]: time="2026-07-25T11:31:36.856Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=12 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ15FTTPD8VPH4KPCQ1G2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:36.833" grpc.time_ms=23.123 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 357.203436] gitaly[1447]: time="2026-07-25T11:31:36.892Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-25T11:31:46.133" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:36.884" grpc.time_ms=8.659 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 357.216914] gitaly[1447]: time="2026-07-25T11:31:36.974Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ19XYTKAS2STMY0EH79P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:06.670" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:36.970" grpc.time_ms=3.118 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 357.235262] gitaly[1447]: time="2026-07-25T11:31:37.048Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=592 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ19XYTKAS2STMY0EH79P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-25T11:32:07.033" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:37.033" grpc.time_ms=14.622 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 357.267039] gitaly[1447]: time="2026-07-25T11:31:37.080Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=1 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ19XYTKAS2STMY0EH79P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-25T11:31:47.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:37.061" grpc.time_ms=18.8 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 357.376054] gitaly[1447]: time="2026-07-25T11:31:37.188Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ19XYTKAS2STMY0EH79P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-25T11:31:47.127" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:37.188" grpc.time_ms=0.691 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 357.410374] gitaly[1447]: time="2026-07-25T11:31:37.221Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ19XYTKAS2STMY0EH79P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-25T11:31:47.142" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:37.203" grpc.time_ms=18.02 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 357.472480] gitaly[1447]: time="2026-07-25T11:31:37.285Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ19XYTKAS2STMY0EH79P grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-25T11:31:37.283" grpc.time_ms=1.471 pid=1447 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 357.520326] gitaly[1447]: time="2026-07-25T11:31:37.333Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ19XYTKAS2STMY0EH79P grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4157 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-25T11:31:37.332" grpc.time_ms=0.756 pid=1447 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 357.550038] gitaly[1447]: time="2026-07-25T11:31:37.346Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ19XYTKAS2STMY0EH79P diskcache=91253d33-dd75-4221-baa6-2449cb55c056 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-25T11:31:47.165" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:37.225" pid=1447 ref=refs/keep-around/008ad05c6b72d3c5b6b0aa9fcd9a124deadf7afe remote_ip="2001:db8:1::1" revision=008ad05c6b72d3c5b6b0aa9fcd9a124deadf7afe span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 357.574385] gitaly[1447]: time="2026-07-25T11:31:37.347Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=120 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5407 command.oublock=8 command.real_time_ms=113 command.spawn_token_fork_ms=2 command.system_time_ms=66 command.user_time_ms=54 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ19XYTKAS2STMY0EH79P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-25T11:31:47.165" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:37.225" grpc.time_ms=121.272 pid=1447 ref=refs/keep-around/008ad05c6b72d3c5b6b0aa9fcd9a124deadf7afe remote_ip="2001:db8:1::1" revision=008ad05c6b72d3c5b6b0aa9fcd9a124deadf7afe span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 357.598266] gitaly[1447]: time="2026-07-25T11:31:37.355Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ19XYTKAS2STMY0EH79P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:07.053" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:37.354" grpc.time_ms=0.795 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=36ae1414ecb17a3920be4f602d1012d76d4248c9' vm-test-run-nixbot-gitlab> gitlab # [ 357.708675] nixbot[4200]: INFO: ::ffff:127.0.0.1:52178 - "GET /api/repos/gitlab/root/test-flake/builds?commit=36ae1414ecb17a3920be4f602d1012d76d4248c9 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=36ae1414ecb17a3920be4f602d1012d76d4248c9', in 0.08 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 357.995802] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/008ad05c6b72d3c5b6b0aa9fcd9a124deadf7afe HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 913 vm-test-run-nixbot-gitlab> gitlab # [ 358.005558] nixbot[4200]: {"ts": "2026-07-25T11:31:37.810Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/008ad05c6b72d3c5b6b0aa9fcd9a124deadf7afe \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 358.014285] nixbot[4200]: {"ts": "2026-07-25T11:31:37.824Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-008ad05c6b72-a6ffe546", "argv0": "bwrap"} vm-test-run-nixbot-gitlab> gitlab # [ 358.128840] gitaly[1447]: time="2026-07-25T11:31:37.940Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=883 command.oublock=0 command.real_time_ms=25 command.spawn_token_fork_ms=5 command.system_time_ms=17 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-25T11:32:07.110" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:37.911" grpc.time_ms=29.377 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 358.166395] systemd[1]: Started Nix Daemon. vm-test-run-nixbot-gitlab> gitlab # [ 358.167743] gitaly[1447]: time="2026-07-25T11:31:37.979Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=10 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ01FCE950ZMHF2RYKPNE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-25T11:32:07.150" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:37.950" grpc.time_ms=28.434 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 358.192733] sidekiq-cluster[2673]: {"severity":"WARN","time":"2026-07-25T11:31:38.005Z","class":"CreatePipelineWorker","correlation_id":"01KYCGQ01FCE950ZMHF2RYKPNE","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"008ad05c6b72d3c5b6b0aa9fcd9a124deadf7afe","after":"36ae1414ecb17a3920be4f602d1012d76d4248c9","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"36ae1414ecb17a3920be4f602d1012d76d4248c9","ref":"refs/heads/master"},"retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 358.204752] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:38.013Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"d8d62198080eb6c9dc83cca7","created_at":"2026-07-25T11:31:36.394Z","trace_propagation_headers":{"sentry-trace":"fc8c5dc0b26f4fdf89361d7b7289c9af-094e23b3b345449d","baggage":"sentry-trace_id=fc8c5dc0b26f4fdf89361d7b7289c9af,sentry-release=Unknown"},"correlation_id":"01KYCGQ01FCE950ZMHF2RYKPNE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:36fb598e8d54ab2600b07e719e39dcea1bce6dad67a1d179d892e53a4478149a","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:36.399Z","job_size_bytes":467,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-d8d62198080eb6c9dc83cca7: done: 1.290741 sec","job_status":"done","queue_duration_s":0.323114,"scheduling_latency_s":0.323139,"gitaly_calls":7,"gitaly_duration_s":0.118694,"redis_calls":33,"redis_duration_s":0.012740999999999999,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000681,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00435,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.006447,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001263,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.262,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.257,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.906388,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":1.290741,"completed_at":"2026-07-25T11:31:38.013Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.261584,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 358.268356] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:38.016Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"012ce76a-eb4c-46fd-ba90-c39d7d43e9bf\"}"],"class":"WebHookWorker","jid":"4bbac8279aa9863c72bcb616","created_at":"2026-07-25T11:31:36.562Z","trace_propagation_headers":{"sentry-trace":"fc8c5dc0b26f4fdf89361d7b7289c9af-094e23b3b345449d","baggage":"sentry-trace_id=fc8c5dc0b26f4fdf89361d7b7289c9af,sentry-release=Unknown"},"correlation_id":"01KYCGQ01FCE950ZMHF2RYKPNE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:342547fc4c67b8f99a6c4715c3e719d5bbd278d043344fe3c5730621e483af2b","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:36.570Z","job_size_bytes":1749,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-4bbac8279aa9863c72bcb616: start","job_status":"start","queue_duration_s":1.445923,"scheduling_latency_s":1.445957} vm-test-run-nixbot-gitlab> gitlab # [ 358.291465] nix-daemon[4535]: accepted connection from pid 4528, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 358.292720] nix-daemon[4535]: accepted connection from pid 4530, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 358.359261] nixbot[4200]: INFO: ::ffff:127.0.0.1:52186 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted vm-test-run-nixbot-gitlab> gitlab # [ 358.401594] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:38.213Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"012ce76a-eb4c-46fd-ba90-c39d7d43e9bf\"}"],"class":"WebHookWorker","jid":"4bbac8279aa9863c72bcb616","created_at":"2026-07-25T11:31:36.562Z","trace_propagation_headers":{"sentry-trace":"fc8c5dc0b26f4fdf89361d7b7289c9af-094e23b3b345449d","baggage":"sentry-trace_id=fc8c5dc0b26f4fdf89361d7b7289c9af,sentry-release=Unknown"},"correlation_id":"01KYCGQ01FCE950ZMHF2RYKPNE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:342547fc4c67b8f99a6c4715c3e719d5bbd278d043344fe3c5730621e483af2b","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:36.570Z","job_size_bytes":1749,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-4bbac8279aa9863c72bcb616: done: 0.19703 sec","job_status":"done","queue_duration_s":1.445923,"scheduling_latency_s":1.445957,"redis_calls":19,"redis_duration_s":0.006534,"redis_read_bytes":31,"redis_write_bytes":5282,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001709,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000816,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3644,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002525,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001484,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.03202115999999933,"cpu_s":0.141775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00066116,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.19703,"completed_at":"2026-07-25T11:31:38.213Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002827,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 358.501376] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:38.216Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"5c24042c43cbd625a0c5a009","created_at":"2026-07-25T11:31:37.366Z","trace_propagation_headers":{"sentry-trace":"e62de8594b4e4b608435e507b8a54697-8100c36cc8514816","baggage":"sentry-trace_id=e62de8594b4e4b608435e507b8a54697,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGQ19XYTKAS2STMY0EH79P","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:37.378Z","job_size_bytes":65,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-5c24042c43cbd625a0c5a009: start","job_status":"start","queue_duration_s":0.838149,"scheduling_latency_s":0.838196} vm-test-run-nixbot-gitlab> gitlab # [ 358.523657] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:38.317Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"5c24042c43cbd625a0c5a009","created_at":"2026-07-25T11:31:37.366Z","trace_propagation_headers":{"sentry-trace":"e62de8594b4e4b608435e507b8a54697-8100c36cc8514816","baggage":"sentry-trace_id=e62de8594b4e4b608435e507b8a54697,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGQ19XYTKAS2STMY0EH79P","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:37.378Z","job_size_bytes":65,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-5c24042c43cbd625a0c5a009: done: 0.100948 sec","job_status":"done","queue_duration_s":0.838149,"scheduling_latency_s":0.838196,"redis_calls":18,"redis_duration_s":0.006821,"redis_read_bytes":271,"redis_write_bytes":2103,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.004026,"redis_feature_flag_read_bytes":264,"redis_feature_flag_write_bytes":1285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001786,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064547,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00041682,"duration_s":0.100948,"completed_at":"2026-07-25T11:31:38.317Z","load_balancing_strategy":"primary","db_duration_s":0.015436,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 358.570840] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:38.320Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"e635d6d8d521014fbe7932a4","created_at":"2026-07-25T11:31:37.380Z","trace_propagation_headers":{"sentry-trace":"e62de8594b4e4b608435e507b8a54697-8100c36cc8514816","baggage":"sentry-trace_id=e62de8594b4e4b608435e507b8a54697,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGQ19XYTKAS2STMY0EH79P","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:37.389Z","job_size_bytes":65,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-e635d6d8d521014fbe7932a4: start","job_status":"start","queue_duration_s":0.930434,"scheduling_latency_s":0.93046} vm-test-run-nixbot-gitlab> gitlab # [ 358.588785] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 102 vm-test-run-nixbot-gitlab> gitlab # [ 358.591131] gitaly[1447]: time="2026-07-25T11:31:38.321Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ2KBWE855MBRSXMPH795 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:38.309" grpc.time_ms=11.308 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 358.604915] gitaly[1447]: time="2026-07-25T11:31:38.412Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ2PM3P4XTF2CHV5SKG5W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:38.397" pid=1447 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 358.615684] gitlab-workhorse[1369]: time="2026-07-25T11:31:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 358.617716] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 89 vm-test-run-nixbot-gitlab> gitlab # [ 358.619644] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:38.410Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"e635d6d8d521014fbe7932a4","created_at":"2026-07-25T11:31:37.380Z","trace_propagation_headers":{"sentry-trace":"e62de8594b4e4b608435e507b8a54697-8100c36cc8514816","baggage":"sentry-trace_id=e62de8594b4e4b608435e507b8a54697,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGQ19XYTKAS2STMY0EH79P","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:37.389Z","job_size_bytes":65,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-e635d6d8d521014fbe7932a4: done: 0.090198 sec","job_status":"done","queue_duration_s":0.930434,"scheduling_latency_s":0.93046,"redis_calls":15,"redis_duration_s":0.005758,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000459,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0016,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003699,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028312,"duration_s":0.090198,"completed_at":"2026-07-25T11:31:38.410Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015342,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 358.668907] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:38.412Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"8e44ce02c9594809d49c84b5","created_at":"2026-07-25T11:31:37.774Z","trace_propagation_headers":{"sentry-trace":"e62de8594b4e4b608435e507b8a54697-8100c36cc8514816","baggage":"sentry-trace_id=e62de8594b4e4b608435e507b8a54697,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGQ19XYTKAS2STMY0EH79P","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:37.782Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8e44ce02c9594809d49c84b5: start","job_status":"start","queue_duration_s":0.630103,"scheduling_latency_s":0.630128} vm-test-run-nixbot-gitlab> gitlab # [ 358.685175] gitaly[1447]: time="2026-07-25T11:31:38.412Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=7 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ2PM3P4XTF2CHV5SKG5W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:38.397" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 358.698964] gitaly[1447]: time="2026-07-25T11:31:38.412Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=7 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ2PM3P4XTF2CHV5SKG5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:38.397" grpc.time_ms=15.45 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=36ae1414ecb17a3920be4f602d1012d76d4248c9' vm-test-run-nixbot-gitlab> gitlab # [ 358.715514] gitaly[1447]: time="2026-07-25T11:31:38.528Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ2TN18GAENHWX4YZTHHW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=196 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:38.518" grpc.time_ms=10.501 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 358.728059] gitaly[1447]: time="2026-07-25T11:31:38.539Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ19XYTKAS2STMY0EH79P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-25T11:32:08.038" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:38.538" grpc.time_ms=1.048 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 358.741074] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 75 vm-test-run-nixbot-gitlab> gitlab # [ 358.769162] nixbot[4200]: INFO: ::ffff:127.0.0.1:52192 - "GET /api/repos/gitlab/root/test-flake/builds?commit=36ae1414ecb17a3920be4f602d1012d76d4248c9 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=36ae1414ecb17a3920be4f602d1012d76d4248c9', in 0.06 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 358.775638] gitaly[1447]: time="2026-07-25T11:31:38.588Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ19XYTKAS2STMY0EH79P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:08.086" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:38.586" grpc.time_ms=2.026 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 358.815654] gitaly[1447]: time="2026-07-25T11:31:38.628Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ19XYTKAS2STMY0EH79P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:08.127" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:38.627" grpc.time_ms=0.716 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 358.829894] gitaly[1447]: time="2026-07-25T11:31:38.638Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ2X331FAYBPXXHMQN8DA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:38.616" pid=1447 remote_ip="2001:db8:1::1" request_sha=f1e6bb59289c4d43fc3699ed3fc4760ee01ef0e9 response_bytes=216 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 358.842232] gitlab-workhorse[1369]: time="2026-07-25T11:31:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 vm-test-run-nixbot-gitlab> gitlab # [ 358.844324] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.54.0" 108 vm-test-run-nixbot-gitlab> gitlab # [ 358.846388] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:38.648Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"8e44ce02c9594809d49c84b5","created_at":"2026-07-25T11:31:37.774Z","trace_propagation_headers":{"sentry-trace":"e62de8594b4e4b608435e507b8a54697-8100c36cc8514816","baggage":"sentry-trace_id=e62de8594b4e4b608435e507b8a54697,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGQ19XYTKAS2STMY0EH79P","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:37.782Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8e44ce02c9594809d49c84b5: done: 0.235716 sec","job_status":"done","queue_duration_s":0.630103,"scheduling_latency_s":0.630128,"gitaly_calls":3,"gitaly_duration_s":0.012447,"redis_calls":48,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.014157,"redis_read_bytes":208,"redis_write_bytes":6803,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.0016,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001838,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.001387,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.000244,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.003225,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000408,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000175,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00368,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.016,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.183154,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.19305286,"duration_s":0.235716,"completed_at":"2026-07-25T11:31:38.648Z","load_balancing_strategy":"primary","db_duration_s":0.038745,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 358.902671] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:38.651Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","4bbac8279aa9863c72bcb616"],"class":"WebHooks::LogExecutionWorker","jid":"42469e7c151bd6667df91863","created_at":"2026-07-25T11:31:38.177Z","trace_propagation_headers":{"sentry-trace":"fc8c5dc0b26f4fdf89361d7b7289c9af-deca584da15b40ea","baggage":"sentry-trace_id=fc8c5dc0b26f4fdf89361d7b7289c9af,sentry-release=Unknown"},"correlation_id":"01KYCGQ01FCE950ZMHF2RYKPNE","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d5e15f9a454727406a5c302fdf365a3a633e1c9b102e057a1733454bb834afbb","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:38.189Z","job_size_bytes":2507,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-42469e7c151bd6667df91863: start","job_status":"start","queue_duration_s":0.461373,"scheduling_latency_s":0.461404} vm-test-run-nixbot-gitlab> gitlab # [ 358.920040] gitaly[1447]: time="2026-07-25T11:31:38.638Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ2X331FAYBPXXHMQN8DA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:38.616" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 358.934211] nix-daemon[4543]: unexpected Nix daemon error: error: interrupted by the user vm-test-run-nixbot-gitlab> gitlab # [ 358.935412] nixbot[4200]: {"ts": "2026-07-25T11:31:38.677Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"} vm-test-run-nixbot-gitlab> gitlab # [ 358.937507] gitaly[1447]: time="2026-07-25T11:31:38.639Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ2X331FAYBPXXHMQN8DA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-25T11:31:38.616" grpc.time_ms=22.382 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 358.952447] gitaly[1447]: time="2026-07-25T11:31:38.758Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ31RCYDDFK5EVJ62GNTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:08.456" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:38.757" grpc.time_ms=0.743 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 358.970714] gitaly[1447]: time="2026-07-25T11:31:38.783Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=594 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=6 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ31RCYDDFK5EVJ62GNTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-25T11:32:08.470" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:38.771" grpc.time_ms=12.454 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 358.985299] gitaly[1447]: time="2026-07-25T11:31:38.796Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ33BA0S9N6CBGBSAE6J5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:08.095" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:38.795" grpc.time_ms=0.659 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 358.998115] gitaly[1447]: time="2026-07-25T11:31:38.802Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ31RCYDDFK5EVJ62GNTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-25T11:31:48.123" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:38.794" grpc.time_ms=8.513 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 359.020415] gitaly[1447]: time="2026-07-25T11:31:38.833Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ31RCYDDFK5EVJ62GNTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-25T11:31:48.162" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:38.833" grpc.time_ms=0.37 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 359.036217] gitaly[1447]: time="2026-07-25T11:31:38.849Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ31RCYDDFK5EVJ62GNTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-25T11:31:48.169" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:38.840" grpc.time_ms=8.906 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 359.055660] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:38.867Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","4bbac8279aa9863c72bcb616"],"class":"WebHooks::LogExecutionWorker","jid":"42469e7c151bd6667df91863","created_at":"2026-07-25T11:31:38.177Z","trace_propagation_headers":{"sentry-trace":"fc8c5dc0b26f4fdf89361d7b7289c9af-deca584da15b40ea","baggage":"sentry-trace_id=fc8c5dc0b26f4fdf89361d7b7289c9af,sentry-release=Unknown"},"correlation_id":"01KYCGQ01FCE950ZMHF2RYKPNE","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d5e15f9a454727406a5c302fdf365a3a633e1c9b102e057a1733454bb834afbb","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:38.189Z","job_size_bytes":2507,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-42469e7c151bd6667df91863: done: 0.216834 sec","job_status":"done","queue_duration_s":0.461373,"scheduling_latency_s":0.461404,"redis_calls":10,"redis_duration_s":0.004579,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001656,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001516,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001407,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.054,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.054,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.176928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00041802,"duration_s":0.216834,"completed_at":"2026-07-25T11:31:38.867Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013047,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 359.104194] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:38.872Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"822bf2c89c45253796f27a8c","created_at":"2026-07-25T11:31:38.515Z","trace_propagation_headers":{"sentry-trace":"e62de8594b4e4b608435e507b8a54697-6b94e091211e4091","baggage":"sentry-trace_id=e62de8594b4e4b608435e507b8a54697,sentry-release=Unknown"},"correlation_id":"01KYCGQ19XYTKAS2STMY0EH79P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:38.522Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-822bf2c89c45253796f27a8c: start","job_status":"start","queue_duration_s":0.349992,"scheduling_latency_s":0.350029} vm-test-run-nixbot-gitlab> gitlab # [ 359.121359] gitaly[1447]: time="2026-07-25T11:31:38.893Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ31RCYDDFK5EVJ62GNTT grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-25T11:31:38.892" grpc.time_ms=0.462 pid=1447 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 359.132542] gitaly[1447]: time="2026-07-25T11:31:38.928Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ31RCYDDFK5EVJ62GNTT grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4157 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-25T11:31:38.926" grpc.time_ms=1.136 pid=1447 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 359.142532] gitaly[1447]: time="2026-07-25T11:31:38.940Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ31RCYDDFK5EVJ62GNTT diskcache=57b32be1-501f-465d-a210-f45ba2914ce9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-25T11:31:48.182" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:38.852" pid=1447 ref=refs/keep-around/36ae1414ecb17a3920be4f602d1012d76d4248c9 remote_ip="2001:db8:1::1" revision=36ae1414ecb17a3920be4f602d1012d76d4248c9 span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 359.154420] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:38.940Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"822bf2c89c45253796f27a8c","created_at":"2026-07-25T11:31:38.515Z","trace_propagation_headers":{"sentry-trace":"e62de8594b4e4b608435e507b8a54697-6b94e091211e4091","baggage":"sentry-trace_id=e62de8594b4e4b608435e507b8a54697,sentry-release=Unknown"},"correlation_id":"01KYCGQ19XYTKAS2STMY0EH79P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:38.522Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-822bf2c89c45253796f27a8c: done: 0.068426 sec","job_status":"done","queue_duration_s":0.349992,"scheduling_latency_s":0.350029,"redis_calls":10,"redis_duration_s":0.0022069999999999998,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000456,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001222,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000529,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038742,"duration_s":0.068426,"completed_at":"2026-07-25T11:31:38.940Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016602,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 359.200586] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:38.955Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"98b381670815aef79b9c20ab","created_at":"2026-07-25T11:31:38.948Z","trace_propagation_headers":{"sentry-trace":"7ae664cb17414d68a788e0b5b8387630-066fd4d3161a4a1e","baggage":"sentry-trace_id=7ae664cb17414d68a788e0b5b8387630,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGQ31RCYDDFK5EVJ62GNTT","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:38.952Z","job_size_bytes":65,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-98b381670815aef79b9c20ab: start","job_status":"start","queue_duration_s":0.002057,"scheduling_latency_s":0.002096} vm-test-run-nixbot-gitlab> gitlab # [ 359.217739] gitaly[1447]: time="2026-07-25T11:31:38.940Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=56 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4665 command.oublock=8 command.real_time_ms=82 command.spawn_token_fork_ms=1 command.system_time_ms=25 command.user_time_ms=31 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ31RCYDDFK5EVJ62GNTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-25T11:31:48.182" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-25T11:31:38.852" grpc.time_ms=88.169 pid=1447 ref=refs/keep-around/36ae1414ecb17a3920be4f602d1012d76d4248c9 remote_ip="2001:db8:1::1" revision=36ae1414ecb17a3920be4f602d1012d76d4248c9 span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 359.232962] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:39.003Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"98b381670815aef79b9c20ab","created_at":"2026-07-25T11:31:38.948Z","trace_propagation_headers":{"sentry-trace":"7ae664cb17414d68a788e0b5b8387630-066fd4d3161a4a1e","baggage":"sentry-trace_id=7ae664cb17414d68a788e0b5b8387630,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGQ31RCYDDFK5EVJ62GNTT","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:38.952Z","job_size_bytes":65,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-98b381670815aef79b9c20ab: done: 0.048483 sec","job_status":"done","queue_duration_s":0.002057,"scheduling_latency_s":0.002096,"redis_calls":7,"redis_duration_s":0.010616,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007322,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003294,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024519,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048483,"completed_at":"2026-07-25T11:31:39.003Z","load_balancing_strategy":"primary","db_duration_s":0.007451,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 359.278107] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:39.005Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"9aea40c4b7868c0102e49f73","created_at":"2026-07-25T11:31:38.954Z","trace_propagation_headers":{"sentry-trace":"7ae664cb17414d68a788e0b5b8387630-066fd4d3161a4a1e","baggage":"sentry-trace_id=7ae664cb17414d68a788e0b5b8387630,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGQ31RCYDDFK5EVJ62GNTT","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:38.959Z","job_size_bytes":65,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-9aea40c4b7868c0102e49f73: start","job_status":"start","queue_duration_s":0.045839,"scheduling_latency_s":0.045865} vm-test-run-nixbot-gitlab> gitlab # [ 359.298104] gitaly[1447]: time="2026-07-25T11:31:38.946Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ31RCYDDFK5EVJ62GNTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:08.644" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:38.945" grpc.time_ms=0.771 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 359.311777] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:39.087Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"9aea40c4b7868c0102e49f73","created_at":"2026-07-25T11:31:38.954Z","trace_propagation_headers":{"sentry-trace":"7ae664cb17414d68a788e0b5b8387630-066fd4d3161a4a1e","baggage":"sentry-trace_id=7ae664cb17414d68a788e0b5b8387630,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGQ31RCYDDFK5EVJ62GNTT","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:38.959Z","job_size_bytes":65,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-9aea40c4b7868c0102e49f73: done: 0.081764 sec","job_status":"done","queue_duration_s":0.045839,"scheduling_latency_s":0.045865,"redis_calls":10,"redis_duration_s":0.005945,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001725,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.00422,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.081764,"completed_at":"2026-07-25T11:31:39.087Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022046,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 359.357909] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:39.097Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"62afaf4606f638a5de9dd11e","created_at":"2026-07-25T11:31:39.084Z","trace_propagation_headers":{"sentry-trace":"7ae664cb17414d68a788e0b5b8387630-066fd4d3161a4a1e","baggage":"sentry-trace_id=7ae664cb17414d68a788e0b5b8387630,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGQ31RCYDDFK5EVJ62GNTT","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.096Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-62afaf4606f638a5de9dd11e: start","job_status":"start","queue_duration_s":0.001538,"scheduling_latency_s":0.001568} vm-test-run-nixbot-gitlab> gitlab # [ 359.374886] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:39 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/36ae1414ecb17a3920be4f602d1012d76d4248c9 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 444 vm-test-run-nixbot-gitlab> gitlab # [ 359.377701] nixbot[4200]: {"ts": "2026-07-25T11:31:39.126Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/36ae1414ecb17a3920be4f602d1012d76d4248c9 \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 359.380624] nixbot[4200]: {"ts": "2026-07-25T11:31:39.134Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-36ae1414ecb1-22ddf057", "argv0": "bwrap"} vm-test-run-nixbot-gitlab> gitlab # [ 359.390970] gitaly[1447]: time="2026-07-25T11:31:39.204Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ31RCYDDFK5EVJ62GNTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-25T11:32:09.002" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:39.203" grpc.time_ms=0.945 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 359.464538] gitaly[1447]: time="2026-07-25T11:31:39.277Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ33BA0S9N6CBGBSAE6J5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-25T11:31:49.248" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:39.268" grpc.time_ms=8.622 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 359.479634] nix-daemon[4535]: accepted connection from pid 4620, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 359.480826] gitaly[1447]: time="2026-07-25T11:31:39.292Z" level=info msg="finished unary call with code OK" catfile.duration_ms=6 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=6 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ31RCYDDFK5EVJ62GNTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:09.085" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:39.286" grpc.time_ms=6.413 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 359.496738] nix-daemon[4535]: accepted connection from pid 4619, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 359.523452] gitaly[1447]: time="2026-07-25T11:31:39.336Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ31RCYDDFK5EVJ62GNTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:09.135" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:39.335" grpc.time_ms=0.859 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 359.538335] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:39.351Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"62afaf4606f638a5de9dd11e","created_at":"2026-07-25T11:31:39.084Z","trace_propagation_headers":{"sentry-trace":"7ae664cb17414d68a788e0b5b8387630-066fd4d3161a4a1e","baggage":"sentry-trace_id=7ae664cb17414d68a788e0b5b8387630,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGQ31RCYDDFK5EVJ62GNTT","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.096Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-62afaf4606f638a5de9dd11e: done: 0.253304 sec","job_status":"done","queue_duration_s":0.001538,"scheduling_latency_s":0.001568,"gitaly_calls":3,"gitaly_duration_s":0.024256,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.017976,"redis_read_bytes":98,"redis_write_bytes":6014,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002892,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002535,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000282,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003039,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.003193,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000476,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002667,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.049,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.169416,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.21941796,"duration_s":0.253304,"completed_at":"2026-07-25T11:31:39.350Z","load_balancing_strategy":"primary","db_duration_s":0.050437,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 359.592897] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:39.352Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"50c22a693751e26ca7e0d530","created_at":"2026-07-25T11:31:39.178Z","trace_propagation_headers":{"sentry-trace":"7ae664cb17414d68a788e0b5b8387630-dee15114759946f6","baggage":"sentry-trace_id=7ae664cb17414d68a788e0b5b8387630,sentry-release=Unknown"},"correlation_id":"01KYCGQ31RCYDDFK5EVJ62GNTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.181Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-50c22a693751e26ca7e0d530: start","job_status":"start","queue_duration_s":0.171083,"scheduling_latency_s":0.17111} vm-test-run-nixbot-gitlab> gitlab # [ 359.609734] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:39.388Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"50c22a693751e26ca7e0d530","created_at":"2026-07-25T11:31:39.178Z","trace_propagation_headers":{"sentry-trace":"7ae664cb17414d68a788e0b5b8387630-dee15114759946f6","baggage":"sentry-trace_id=7ae664cb17414d68a788e0b5b8387630,sentry-release=Unknown"},"correlation_id":"01KYCGQ31RCYDDFK5EVJ62GNTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.181Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-50c22a693751e26ca7e0d530: done: 0.035996 sec","job_status":"done","queue_duration_s":0.171083,"scheduling_latency_s":0.17111,"redis_calls":7,"redis_duration_s":0.00127,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000333,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035996,"completed_at":"2026-07-25T11:31:39.388Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005932,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 359.653741] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:39 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/008ad05c6b72d3c5b6b0aa9fcd9a124deadf7afe HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 713 vm-test-run-nixbot-gitlab> gitlab # [ 359.656346] nixbot[4200]: {"ts": "2026-07-25T11:31:39.447Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/008ad05c6b72d3c5b6b0aa9fcd9a124deadf7afe \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 359.659164] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:39.419Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3f09daae6c54b5353336f90f","created_at":"2026-07-25T11:31:39.412Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-82f83ed0861c4587","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.417Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3f09daae6c54b5353336f90f: start","job_status":"start","queue_duration_s":0.001942,"scheduling_latency_s":0.001997} vm-test-run-nixbot-gitlab> gitlab # [ 359.714110] gitaly[1447]: time="2026-07-25T11:31:39.527Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ3T5THQ7WRJGC3451A39 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:09.025" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:39.526" grpc.time_ms=1.046 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=36ae1414ecb17a3920be4f602d1012d76d4248c9' vm-test-run-nixbot-gitlab> gitlab # [ 359.777344] gitaly[1447]: time="2026-07-25T11:31:39.590Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ3T5THQ7WRJGC3451A39 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-25T11:31:49.224" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:39.575" grpc.time_ms=15.049 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 359.847869] nixbot[4200]: INFO: ::ffff:127.0.0.1:52198 - "GET /api/repos/gitlab/root/test-flake/builds?commit=36ae1414ecb17a3920be4f602d1012d76d4248c9 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=36ae1414ecb17a3920be4f602d1012d76d4248c9', in 0.07 seconds) vm-test-run-nixbot-gitlab> (finished: subtest: push delivers the webhook, in 4.30 seconds) vm-test-run-nixbot-gitlab> subtest: eval and build post commit statuses vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/36ae1414ecb17a3920be4f602d1012d76d4248c9/statuses vm-test-run-nixbot-gitlab> gitlab # [ 359.851231] gitaly[1447]: time="2026-07-25T11:31:39.664Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ33BA0S9N6CBGBSAE6J5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-25T11:31:39.663" grpc.time_ms=0.546 pid=1447 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 359.866972] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:39 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/008ad05c6b72d3c5b6b0aa9fcd9a124deadf7afe HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 213 vm-test-run-nixbot-gitlab> gitlab # [ 359.870745] nixbot[4200]: {"ts": "2026-07-25T11:31:39.675Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/008ad05c6b72d3c5b6b0aa9fcd9a124deadf7afe \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 359.904187] gitaly[1447]: time="2026-07-25T11:31:39.717Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ33BA0S9N6CBGBSAE6J5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-25T11:31:39.716" grpc.time_ms=0.497 pid=1447 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 359.933962] gitaly[1447]: time="2026-07-25T11:31:39.747Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ33BA0S9N6CBGBSAE6J5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-25T11:31:39.746" grpc.time_ms=0.483 pid=1447 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 359.952087] gitaly[1447]: time="2026-07-25T11:31:39.758Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ33BA0S9N6CBGBSAE6J5 diskcache=16a6b4cd-4f4d-4ede-931c-9cac4c5f72c8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-25T11:32:09.118" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:39.618" pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 359.966885] gitaly[1447]: time="2026-07-25T11:31:39.759Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=123 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7314 command.oublock=0 command.real_time_ms=133 command.spawn_token_fork_ms=1 command.system_time_ms=53 command.user_time_ms=70 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ33BA0S9N6CBGBSAE6J5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-25T11:32:09.118" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:39.618" grpc.time_ms=140.333 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 359.983914] gitaly[1447]: time="2026-07-25T11:31:39.773Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ41H6BX30QHY33K4ZGXP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:09.271" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:39.772" grpc.time_ms=0.803 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/36ae1414ecb17a3920be4f602d1012d76d4248c9/statuses, in 0.14 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 359.999371] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/36ae1414ecb17a3920be4f602d1012d76d4248c9/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 109 vm-test-run-nixbot-gitlab> gitlab # [ 360.089310] gitaly[1447]: time="2026-07-25T11:31:39.902Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ33BA0S9N6CBGBSAE6J5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-25T11:32:09.100" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:39.900" grpc.time_ms=1.703 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 360.162487] gitaly[1447]: time="2026-07-25T11:31:39.974Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ33BA0S9N6CBGBSAE6J5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:09.171" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:39.972" grpc.time_ms=1.491 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 360.261845] gitaly[1447]: time="2026-07-25T11:31:40.074Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ33BA0S9N6CBGBSAE6J5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:10.072" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:40.072" grpc.time_ms=1.569 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 360.300624] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:40.095Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"8ed182de790f37823ead8c1f","created_at":"2026-07-25T11:31:40.091Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"3f09daae6c54b5353336f90f","job_size_bytes":3,"pid":2673,"job_status":"deduplicated","message":"PipelineProcessWorker JID-8ed182de790f37823ead8c1f: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} vm-test-run-nixbot-gitlab> gitlab # [ 360.334410] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:40.099Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3f09daae6c54b5353336f90f","created_at":"2026-07-25T11:31:39.412Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-82f83ed0861c4587","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.417Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2673,"job_status":"rescheduled","message":"PipelineProcessWorker JID-3f09daae6c54b5353336f90f: rescheduled"} vm-test-run-nixbot-gitlab> gitlab # [ 360.360392] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:40.112Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3f09daae6c54b5353336f90f","created_at":"2026-07-25T11:31:39.412Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-82f83ed0861c4587","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.417Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3f09daae6c54b5353336f90f: done: 0.693278 sec","job_status":"done","queue_duration_s":0.001942,"scheduling_latency_s":0.001997,"gitaly_calls":4,"gitaly_duration_s":0.161222,"redis_calls":104,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.036466,"redis_read_bytes":264,"redis_write_bytes":24010,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002203,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003223,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.008202,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.005124,"redis_queues_read_bytes":24,"redis_queues_write_bytes":12389,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.00863,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.001201,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001788,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003892,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.089,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.03,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.035,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.046,"db_ci_duration_s":0.043,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.439884,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.61415462,"duration_s":0.693278,"completed_at":"2026-07-25T11:31:40.112Z","load_balancing_strategy":"primary","db_duration_s":0.092911,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 360.429121] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:40.115Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"83358ccbfdf32ce0b0fde196","created_at":"2026-07-25T11:31:39.587Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.597Z","job_size_bytes":23,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-83358ccbfdf32ce0b0fde196: start","job_status":"start","queue_duration_s":0.517798,"scheduling_latency_s":0.517839} vm-test-run-nixbot-gitlab> gitlab # [ 360.471774] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:40.284Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"83358ccbfdf32ce0b0fde196","created_at":"2026-07-25T11:31:39.587Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.597Z","job_size_bytes":23,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-83358ccbfdf32ce0b0fde196: done: 0.168717 sec","job_status":"done","queue_duration_s":0.517798,"scheduling_latency_s":0.517839,"redis_calls":17,"redis_duration_s":0.005785,"redis_read_bytes":15,"redis_write_bytes":3277,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001371,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":560,"redis_queues_calls":2,"redis_queues_duration_s":0.000513,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1627,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002306,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000895,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0007,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":14,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":13,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.13251,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043562,"duration_s":0.168717,"completed_at":"2026-07-25T11:31:40.284Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027793,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 360.528574] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:40.286Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"d6dea929d18020e6ef474fd4","created_at":"2026-07-25T11:31:40.270Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-71269293c6f047ed","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:d68b855d2268ad8bd58cdc1cde72e5f665787ecb4f0c79cf4ef0a4b162d6b2a3","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:40.273Z","job_size_bytes":464,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-d6dea929d18020e6ef474fd4: start","job_status":"start","queue_duration_s":0.013018,"scheduling_latency_s":0.013051} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/36ae1414ecb17a3920be4f602d1012d76d4248c9/statuses vm-test-run-nixbot-gitlab> gitlab # [ 360.701946] sidekiq-cluster[2673]: {"severity":"WARN","time":"2026-07-25T11:31:40.513Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"d6dea929d18020e6ef474fd4","created_at":"2026-07-25T11:31:40.270Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-71269293c6f047ed","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:d68b855d2268ad8bd58cdc1cde72e5f665787ecb4f0c79cf4ef0a4b162d6b2a3","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:40.273Z","job_size_bytes":464,"pid":2673,"sidekiq_tid":"m3y5","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-d6dea929d18020e6ef474fd4: fail: 0.073848 sec","job_status":"fail","queue_duration_s":0.013018,"scheduling_latency_s":0.013051,"redis_calls":7,"redis_duration_s":0.002459,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001736,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000723,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.073848,"completed_at":"2026-07-25T11:31:40.360Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/w9mp1gyfs2qdv12r0ji70idzndmxwz2s-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/p8ylyr0sj9sxcknqcpja13nypqixk2dp-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/dndr7b4a17339dsm67vzb6nv4p2kp82q-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/zx6ndpscxs64hnidz69ab430ccms6rak-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.013986,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 361.221285] gitaly[1447]: time="2026-07-25T11:31:40.984Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ56D1R9C3ZRJC9553PV2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:10.479" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:40.980" grpc.time_ms=1.284 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/36ae1414ecb17a3920be4f602d1012d76d4248c9/statuses, in 0.24 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 361.234392] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:41 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/36ae1414ecb17a3920be4f602d1012d76d4248c9/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 150 vm-test-run-nixbot-gitlab> gitlab # [ 361.236971] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:40.519Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"7b6d8519edf3694895049a15","created_at":"2026-07-25T11:31:39.767Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.778Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-7b6d8519edf3694895049a15: start","job_status":"start","queue_duration_s":0.741078,"scheduling_latency_s":0.741143} vm-test-run-nixbot-gitlab> gitlab # [ 361.253714] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:40.602Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"7b6d8519edf3694895049a15","created_at":"2026-07-25T11:31:39.767Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.778Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-7b6d8519edf3694895049a15: done: 0.083122 sec","job_status":"done","queue_duration_s":0.741078,"scheduling_latency_s":0.741143,"redis_calls":10,"redis_duration_s":0.005332,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000904,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003227,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001201,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001265,"duration_s":0.083122,"completed_at":"2026-07-25T11:31:40.602Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009381,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 361.298517] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:40.606Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"58f8997de2d78859e0773ed7","created_at":"2026-07-25T11:31:39.803Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.815Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-58f8997de2d78859e0773ed7: start","job_status":"start","queue_duration_s":0.790625,"scheduling_latency_s":0.790678} vm-test-run-nixbot-gitlab> gitlab # [ 361.314996] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:40.692Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"58f8997de2d78859e0773ed7","created_at":"2026-07-25T11:31:39.803Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.815Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-58f8997de2d78859e0773ed7: done: 0.086579 sec","job_status":"done","queue_duration_s":0.790625,"scheduling_latency_s":0.790678,"redis_calls":10,"redis_duration_s":0.00448,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00089,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002729,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000861,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055049,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00086796,"duration_s":0.086579,"completed_at":"2026-07-25T11:31:40.692Z","load_balancing_strategy":"primary","db_duration_s":0.015017,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 361.359297] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:40.697Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"1fa51c142a45e8e1b104efe5","created_at":"2026-07-25T11:31:39.833Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.846Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-1fa51c142a45e8e1b104efe5: start","job_status":"start","queue_duration_s":0.851179,"scheduling_latency_s":0.851247} vm-test-run-nixbot-gitlab> gitlab # [ 361.375193] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:40.806Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"1fa51c142a45e8e1b104efe5","created_at":"2026-07-25T11:31:39.833Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.846Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-1fa51c142a45e8e1b104efe5: done: 0.10937 sec","job_status":"done","queue_duration_s":0.851179,"scheduling_latency_s":0.851247,"redis_calls":13,"redis_duration_s":0.007588,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000923,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003331,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002244,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00109,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.072935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00075604,"duration_s":0.10937,"completed_at":"2026-07-25T11:31:40.806Z","load_balancing_strategy":"primary","db_duration_s":0.011239,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 361.464973] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:40.810Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"5fc6f7bd7a458e7b6df9d0b5","created_at":"2026-07-25T11:31:39.847Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.858Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-5fc6f7bd7a458e7b6df9d0b5: start","job_status":"start","queue_duration_s":0.951275,"scheduling_latency_s":0.951323} vm-test-run-nixbot-gitlab> gitlab # [ 361.493948] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:40.917Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"5fc6f7bd7a458e7b6df9d0b5","created_at":"2026-07-25T11:31:39.847Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.858Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-5fc6f7bd7a458e7b6df9d0b5: done: 0.106842 sec","job_status":"done","queue_duration_s":0.951275,"scheduling_latency_s":0.951323,"redis_calls":10,"redis_duration_s":0.009275,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002391,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00586,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.072006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0011422,"duration_s":0.106842,"completed_at":"2026-07-25T11:31:40.916Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013849,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 361.546704] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:40.920Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"a63a1948dfc468ba2f6f7b73","created_at":"2026-07-25T11:31:39.860Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.870Z","job_size_bytes":19,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-a63a1948dfc468ba2f6f7b73: start","job_status":"start","queue_duration_s":1.050689,"scheduling_latency_s":1.050742} vm-test-run-nixbot-gitlab> gitlab # [ 361.563607] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:41.007Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"a63a1948dfc468ba2f6f7b73","created_at":"2026-07-25T11:31:39.860Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.870Z","job_size_bytes":19,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-a63a1948dfc468ba2f6f7b73: done: 0.087191 sec","job_status":"done","queue_duration_s":1.050689,"scheduling_latency_s":1.050742,"redis_calls":11,"redis_duration_s":0.008809,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000702,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003144,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004963,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0008761,"duration_s":0.087191,"completed_at":"2026-07-25T11:31:41.007Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008964,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 361.611019] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:41.011Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"35791ea99bbae9c2f11ee8e7","created_at":"2026-07-25T11:31:39.875Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.884Z","job_size_bytes":65,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-35791ea99bbae9c2f11ee8e7: start","job_status":"start","queue_duration_s":1.127138,"scheduling_latency_s":1.127183} vm-test-run-nixbot-gitlab> gitlab # [ 361.629496] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:41.115Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"35791ea99bbae9c2f11ee8e7","created_at":"2026-07-25T11:31:39.875Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.884Z","job_size_bytes":65,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-35791ea99bbae9c2f11ee8e7: done: 0.10439 sec","job_status":"done","queue_duration_s":1.127138,"scheduling_latency_s":1.127183,"redis_calls":10,"redis_duration_s":0.01144,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001338,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007603,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002499,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.068071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00115624,"duration_s":0.10439,"completed_at":"2026-07-25T11:31:41.115Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009528,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 361.676441] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:41.119Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"8541df023d6b6c7e54a102c5","created_at":"2026-07-25T11:31:39.885Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.888Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8541df023d6b6c7e54a102c5: start","job_status":"start","queue_duration_s":1.230607,"scheduling_latency_s":1.23066} vm-test-run-nixbot-gitlab> gitlab # [ 361.693311] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:41.185Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"8541df023d6b6c7e54a102c5","created_at":"2026-07-25T11:31:39.885Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.888Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8541df023d6b6c7e54a102c5: done: 0.06645 sec","job_status":"done","queue_duration_s":1.230607,"scheduling_latency_s":1.23066,"redis_calls":7,"redis_duration_s":0.002977,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002029,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000948,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04906,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.06645,"completed_at":"2026-07-25T11:31:41.185Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007912,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 361.736848] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:41.192Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"8548e0818fbe2f31fc0327d5","created_at":"2026-07-25T11:31:39.976Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.989Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-8548e0818fbe2f31fc0327d5: start","job_status":"start","queue_duration_s":1.203176,"scheduling_latency_s":1.203213} vm-test-run-nixbot-gitlab> gitlab # [ 361.753386] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:41.256Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"8548e0818fbe2f31fc0327d5","created_at":"2026-07-25T11:31:39.976Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:39.989Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-8548e0818fbe2f31fc0327d5: done: 0.064022 sec","job_status":"done","queue_duration_s":1.203176,"scheduling_latency_s":1.203213,"redis_calls":10,"redis_duration_s":0.003348,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000644,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001942,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000762,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00064872,"duration_s":0.064022,"completed_at":"2026-07-25T11:31:41.256Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012924,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 361.799103] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:41.258Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"91c6b56867a4b75e10528ae6","created_at":"2026-07-25T11:31:39.990Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:40.001Z","job_size_bytes":6,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-91c6b56867a4b75e10528ae6: start","job_status":"start","queue_duration_s":1.257018,"scheduling_latency_s":1.257048} vm-test-run-nixbot-gitlab> gitlab # [ 361.815340] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:41.338Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"91c6b56867a4b75e10528ae6","created_at":"2026-07-25T11:31:39.990Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:40.001Z","job_size_bytes":6,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-91c6b56867a4b75e10528ae6: done: 0.080348 sec","job_status":"done","queue_duration_s":1.257018,"scheduling_latency_s":1.257048,"redis_calls":10,"redis_duration_s":0.004845,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000809,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003069,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000967,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00091292,"duration_s":0.080348,"completed_at":"2026-07-25T11:31:41.338Z","load_balancing_strategy":"primary","db_duration_s":0.008134,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 361.859635] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:41.341Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"1399791e0d566245b8c116fe","created_at":"2026-07-25T11:31:40.100Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:40.103Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-1399791e0d566245b8c116fe: start","job_status":"start","queue_duration_s":1.237802,"scheduling_latency_s":1.237829} vm-test-run-nixbot-gitlab> gitlab # [ 361.875608] gitaly[1447]: time="2026-07-25T11:31:41.572Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ33BA0S9N6CBGBSAE6J5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-25T11:32:11.068" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:41.569" grpc.time_ms=2.486 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 361.888644] gitaly[1447]: time="2026-07-25T11:31:41.688Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ33BA0S9N6CBGBSAE6J5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:11.186" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:41.687" grpc.time_ms=1.129 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 361.908565] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:41.717Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"1399791e0d566245b8c116fe","created_at":"2026-07-25T11:31:40.100Z","trace_propagation_headers":{"sentry-trace":"3130a65ff1b3433a98d3a956c04800ca-e42c47527f204571","baggage":"sentry-trace_id=3130a65ff1b3433a98d3a956c04800ca,sentry-release=Unknown"},"correlation_id":"01KYCGQ33BA0S9N6CBGBSAE6J5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:40.103Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-1399791e0d566245b8c116fe: done: 0.376377 sec","job_status":"done","queue_duration_s":1.237802,"scheduling_latency_s":1.237829,"gitaly_calls":2,"gitaly_duration_s":0.021835,"redis_calls":19,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.006572,"redis_read_bytes":42,"redis_write_bytes":1891,"redis_cache_calls":8,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002538,"redis_cache_read_bytes":16,"redis_cache_write_bytes":735,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002132,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.0003,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001602,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":20,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":20,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":9,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.019,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.320742,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.32343256,"duration_s":0.376377,"completed_at":"2026-07-25T11:31:41.717Z","load_balancing_strategy":"primary","db_duration_s":0.044007,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/36ae1414ecb17a3920be4f602d1012d76d4248c9/statuses vm-test-run-nixbot-gitlab> gitlab # [ 362.430391] gitaly[1447]: time="2026-07-25T11:31:42.242Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ6DFG4RXQ6QTNRCMK73F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:12.039" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:42.240" grpc.time_ms=1.391 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/36ae1414ecb17a3920be4f602d1012d76d4248c9/statuses, in 0.25 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 362.486495] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:42 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/36ae1414ecb17a3920be4f602d1012d76d4248c9/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 170 vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/36ae1414ecb17a3920be4f602d1012d76d4248c9/statuses vm-test-run-nixbot-gitlab> gitlab # [ 363.650294] gitaly[1447]: time="2026-07-25T11:31:43.462Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ7KX4WHMZDD1PVZEH9EP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:13.160" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:43.461" grpc.time_ms=1.187 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/36ae1414ecb17a3920be4f602d1012d76d4248c9/statuses, in 0.24 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 363.724819] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/36ae1414ecb17a3920be4f602d1012d76d4248c9/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 178 vm-test-run-nixbot-gitlab> gitlab # [ 364.034040] gitaly[1447]: time="2026-07-25T11:31:43.846Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ7ZSKW9DYR6K8PC7RNTW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:13.544" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:43.845" grpc.time_ms=1.215 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 364.132361] gitaly[1447]: time="2026-07-25T11:31:43.945Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ7ZSKW9DYR6K8PC7RNTW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-25T11:31:53.255" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:43.926" grpc.time_ms=18.998 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 364.224715] sidekiq-cluster[2673]: {"severity":"INFO","time":"2026-07-25T11:31:44.035Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"53babb5893c2f5f1de60bfc4","created_at":"2026-07-25T11:31:44.019Z","trace_propagation_headers":{"sentry-trace":"ecb613e36cc145818dde192a689309f7-c82836eef15548e8","baggage":"sentry-trace_id=ecb613e36cc145818dde192a689309f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KYCGQ7ZSKW9DYR6K8PC7RNTW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-25T11:31:44.028Z","job_size_bytes":3,"pid":2673,"sidekiq_tid":"mamd","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-53babb5893c2f5f1de60bfc4: start","job_status":"start","queue_duration_s":0.003137,"scheduling_latency_s":0.003207} vm-test-run-nixbot-gitlab> gitlab # [ 364.273724] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:44 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/36ae1414ecb17a3920be4f602d1012d76d4248c9 HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 320 vm-test-run-nixbot-gitlab> gitlab # [ 364.281165] nixbot[4200]: {"ts": "2026-07-25T11:31:44.060Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/36ae1414ecb17a3920be4f602d1012d76d4248c9 \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 364.373919] gitaly[1447]: time="2026-07-25T11:31:44.187Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ8A3RX94ZMCG6HNB17EG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:14.184" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:44.185" grpc.time_ms=1.44 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 364.459896] gitaly[1447]: time="2026-07-25T11:31:44.272Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ8A3RX94ZMCG6HNB17EG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-25T11:31:54.198" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:44.259" grpc.time_ms=13.472 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 364.542000] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:44 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/36ae1414ecb17a3920be4f602d1012d76d4248c9 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 287 vm-test-run-nixbot-gitlab> gitlab # [ 364.556633] nixbot[4200]: {"ts": "2026-07-25T11:31:44.365Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/36ae1414ecb17a3920be4f602d1012d76d4248c9 \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 364.681466] gitaly[1447]: time="2026-07-25T11:31:44.493Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ8M1V9Z9G4M50Y6ECTNM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:14.291" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:44.491" grpc.time_ms=1.349 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/36ae1414ecb17a3920be4f602d1012d76d4248c9/statuses vm-test-run-nixbot-gitlab> gitlab # [ 364.782318] gitaly[1447]: time="2026-07-25T11:31:44.595Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KYCGQ8M1V9Z9G4M50Y6ECTNM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-25T11:31:54.227" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-25T11:31:44.578" grpc.time_ms=15.847 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 364.876495] gitaly[1447]: time="2026-07-25T11:31:44.689Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KYCGQ8TQ4WP9NRRJ90A722S3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-25T11:32:14.386" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-25T11:31:44.687" grpc.time_ms=1.296 pid=1447 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 364.924942] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:44 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/36ae1414ecb17a3920be4f602d1012d76d4248c9 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 351 vm-test-run-nixbot-gitlab> gitlab # [ 364.930445] nixbot[4200]: {"ts": "2026-07-25T11:31:44.739Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/36ae1414ecb17a3920be4f602d1012d76d4248c9 \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 364.959890] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/07/25:11:31:44 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/36ae1414ecb17a3920be4f602d1012d76d4248c9/statuses HTTP/1.1" 200 1328 "" "curl/8.20.0" 171 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/36ae1414ecb17a3920be4f602d1012d76d4248c9/statuses, in 0.24 seconds) vm-test-run-nixbot-gitlab> (finished: subtest: eval and build post commit statuses, in 5.11 seconds) vm-test-run-nixbot-gitlab> (finished: run the VM test script, in 434.15 seconds) vm-test-run-nixbot-gitlab> test script finished in 434.21s vm-test-run-nixbot-gitlab> cleanup vm-test-run-nixbot-gitlab> kill QemuMachine (pid 13) vm-test-run-nixbot-gitlab> gitlab # qemu-system-aarch64: terminating on signal 15 from pid 6 (/nix/store/lqn6mbgzzdrqq2qkwddcmxj9z6amdd86-python3-3.13.13/bin/python3.13) vm-test-run-nixbot-gitlab> (finished: cleanup, in 1.84 seconds) vm-test-run-nixbot-gitlab> additionally exposed symbols: vm-test-run-nixbot-gitlab> gitlab, vm-test-run-nixbot-gitlab> vlan1, vm-test-run-nixbot-gitlab> start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-07-25T11:31:35.179Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '36ae1414ecb17a3920be4f602d1012d76d4248c9', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': [{'level': 'warning', 'message': "unable to download 'https://cache.nixos.org/…': Could not resolve hostname (6) Could not resolve host: cache.nixos.org", 'count': 1}], 'error': None, 'created_at': '2026-07-25T11:31:38.666383Z', 'started_at': '2026-07-25T11:31:39.129451Z', 'finished_at': None}] vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '36ae1414ecb17a3920be4f602d1012d76d4248c9', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-25T11:31:39.068Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '36ae1414ecb17a3920be4f602d1012d76d4248c9', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-25T11:31:39.068Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '36ae1414ecb17a3920be4f602d1012d76d4248c9', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-25T11:31:39.068Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '36ae1414ecb17a3920be4f602d1012d76d4248c9', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-25T11:31:39.068Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '36ae1414ecb17a3920be4f602d1012d76d4248c9', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-25T11:31:39.068Z', 'started_at': None, 'finished_at': '2026-07-25T11:31:43.997Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '36ae1414ecb17a3920be4f602d1012d76d4248c9', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-07-25T11:31:44.316Z', 'started_at': None, 'finished_at': '2026-07-25T11:31:44.685Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] post-build step Upload coverage to codecov: ok Skipping codecov: project=Mic92/nixbot attr=default.checks.aarch64-linux.nixbot-gitlab